3 ms·

I see a lot of parallels between these ECS discussions and those had by DBAs working on SQL databases. Why not use a RDBMS engine, perhaps specially tuned, to

I see a lot of parallels between these ECS discussions and those had by DBAs working on SQL databases. Why not use a RDBMS engine, perhaps specially tuned, to handle your game's data? Joining arbitrary tuples together at runtime as fast as possible has been the fitness function of every commercial RDBMS vendor since inception. Latency incurred by the network hop and disk IO is probably helping to maintain an illusion that these engines are somehow fundamentally incompatible with gaming use cases.

animal531 2y ago
I've been working on my own ECS system and can say that you're not wrong, most of it is indeed a data problem (and going past DB into just memory allocation and management). Games in truth don't even have that much data, for example you might have 1-2k entities, obstacles etc. However unfortunately I haven't found any existing DB type solutions that work for the gaming use case, even when looking at in-memory only solutions. Real world business solutions have two major constraints, a) Do a bit of work as fast as possible and b) Do a lot of those bits as fast as possible. They have additional 'pros' where you can always potentially add more hardware to any problem to achieve that throughput. Gaming uses both of those as well but for the second case it comes at it from c) Always obey the framerate. For 60fps you have 16ms to do everything and of that you might have only 8ms available for code. Even so that might seem a lot, you could for example do 10 billion adds in that time. But throw in a few loops, conditionals and suddenly it means that for a few thousand entities you can do almost no data transformation, copying etc. Everything needs to be in place and ready to access as is. Existing DB's are about storing data and delivering it, what we need instead is a dynamic memory manager that can move your data around in memory as needed so that elements that are currently being processed together have some advantageous layout and/or are grouped correctly for some process.
the_real_cher 2y ago
I know databases cache which could potentially solve at least some of the disk IO latency but I'd be curious to see if a network hop on a local host would add too much overhead I know you can actually use SQL lite in file mode which would eliminate both of these issues.
kelseyfrog 2y ago
That's because ECS is a database. I'm not exaggerating. "[ECS] is a database" appears in Scott Bilias's seminal talk on ECS. As Bilas put it, ECS is a data-driven, in-memory columnar store optimized for hot-path queries over components. The reason OOP-based games have the problems they do is because they're essentially solving an ontological problem taxonomically. This mismatch over-complicates the solution. 1. Slide 12 https://www.gamedevs.org/uploads/data-driven-game-object-system.pdf https://www.gamedevs.org/uploads/data-driven-game-object-sys...
- bob1029 2y ago
  > The reason OOP-based games have the problems they do is because they're essentially solving an ontological problem taxonomically. I think this issue is not constrained to OOP-based games. Many business models are not possible to express directly (cleanly) using a heirarchy of OOP types due to concerns around circular dependencies and serialization.
dragonwriter 2y ago
ECS engines (well, the entity storage and query parts) are essentially purpose-built (usually, column-oriented, I believe) in-memory, embedded RDBMS engines, already.

animal531 2y ago

I've been working on my own ECS system and can say that you're not wrong, most of it is indeed a data problem (and going past DB into just memory allocation and management). Games in truth don't even have that much data, for example you might have 1-2k entities, obstacles etc. However unfortunately I haven't found any existing DB type solutions that work for the gaming use case, even when looking at in-memory only solutions. Real world business solutions have two major constraints, a) Do a bit of work as fast as possible and b) Do a lot of those bits as fast as possible. They have additional 'pros' where you can always potentially add more hardware to any problem to achieve that throughput. Gaming uses both of those as well but for the second case it comes at it from c) Always obey the framerate. For 60fps you have 16ms to do everything and of that you might have only 8ms available for code. Even so that might seem a lot, you could for example do 10 billion adds in that time. But throw in a few loops, conditionals and suddenly it means that for a few thousand entities you can do almost no data transformation, copying etc. Everything needs to be in place and ready to access as is. Existing DB's are about storing data and delivering it, what we need instead is a dynamic memory manager that can move your data around in memory as needed so that elements that are currently being processed together have some advantageous layout and/or are grouped correctly for some process.

the_real_cher 2y ago

I know databases cache which could potentially solve at least some of the disk IO latency but I'd be curious to see if a network hop on a local host would add too much overhead I know you can actually use SQL lite in file mode which would eliminate both of these issues.

kelseyfrog 2y ago

That's because ECS is a database. I'm not exaggerating. "[ECS] is a database" appears in Scott Bilias's seminal talk on ECS. As Bilas put it, ECS is a data-driven, in-memory columnar store optimized for hot-path queries over components. The reason OOP-based games have the problems they do is because they're essentially solving an ontological problem taxonomically. This mismatch over-complicates the solution. 1. Slide 12 https://www.gamedevs.org/uploads/data-driven-game-object-system.pdf https://www.gamedevs.org/uploads/data-driven-game-object-sys...

bob1029 2y ago
> The reason OOP-based games have the problems they do is because they're essentially solving an ontological problem taxonomically. I think this issue is not constrained to OOP-based games. Many business models are not possible to express directly (cleanly) using a heirarchy of OOP types due to concerns around circular dependencies and serialization.

dragonwriter 2y ago

ECS engines (well, the entity storage and query parts) are essentially purpose-built (usually, column-oriented, I believe) in-memory, embedded RDBMS engines, already.