8 ms·

Flecs – A fast entity component system for C and C++

200 points by apitman 4y ago

sowbug 4y ago
I read the front page of this site, and I still have a general question about ECSes. For what it's worth, I'm developing a DAW (digital audio workstation) in Rust. It seems that ECS helps compose "things" (an entity) out of groups of data (each a component), and then operates upon combinations of those components (using systems). My domain is organized around behavior (many musical instruments implementing a Rust trait or interface, such as MIDI), but the data constituting each instrument is totally different. So the behaviors are the same, but the data is different. It seems that ECS isn't a fit, which is a shame because the implementations I've seen offer cool features like easy parallel processing, querying, and so on. Is there a design pattern like ECS but for common behaviors instead of common data? Or am I being dense and not seeing how ECS actually is a good fit for this situation?
- ajmmertens 4y ago
  The way you frame the question already suggests a solution, and one that indeed doesn't fit ECS that well. That doesn't mean you couldn't design a DAW using ECS, but you'd have to redesign how it works. For example: - create custom components for different instruments - create systems for specific instruments that match the necessary components - write the result to a common component (e.g. "ChannelOutput") You could then add components for EQ, effects, routing, ... and introduce systems that match those. The advantage of this approach would be that you end up with a bunch of loosely coupled systems (for instruments, effects, mixing, ...) and an in-memory representation that's cache & parallelization friendly. One other datapoint would be that Unreal's animation sequencer uses an ECS design, which has a lot of things in common with a DAW. Glossing over lots of details here, you can't of course design a DAW in one paragraph.
  - sowbug 4y ago
    Thanks (and thanks especially for responding to what should have been a Stack Overflow question). I do see potential for certain ECS systems that span instruments, like DCA, effects wet/dry, routing, and mute. That makes sense. Your suggestion for component/system per instrument was what I felt ECS was asking me to do, but the libraries I investigated seemed to contemplate dozens but not hundreds of components, and that's where I began to think I was putting the cart before the horse. I had an epiphany a while ago that a modern interactive DAW is a video game, except parts of it run at 44,100 fps. And even the "except" part is misleading, because games have sound as well. That opened up my mind to game development as another domain to steal designs from.
indigoabstract 4y ago
This looks pretty well polished and maintained, I wasn't aware of it. So far, I've used an alternative ECS called entt: https://github.com/skypjack/entt https://github.com/skypjack/entt https://www.libhunt.com/compare-flecs-vs-entt?ref=compare https://www.libhunt.com/compare-flecs-vs-entt?ref=compare
- pphysch 4y ago
  EnTT is quite nice because it's more than just a ECS (if you want), it also includes libraries for a variety of other features like resource management. It's more of a "game engine" library, which is fantastic for projects that don't need to muck about in a "game studio" like Unity/Unreal (e.g. for games or simulations that don't require traditional level design).
  - ajmmertens 4y ago
    The same goes for Flecs: - it has tools like https://github.com/flecs-hub/explorer https://github.com/flecs-hub/explorer that work with any Flecs project - it has builtin support for hierarchies/prefabs and more (https://ajmmertens.medium.com/building-games-in-ecs-with-entity-relationships-657275ba2c6c https://ajmmertens.medium.com/building-games-in-ecs-with-ent...) - it has optional support for systems/scheduling - it has an optional reflection framework - it has an optional DSL for describing assets/scenes, with an online playground to try it out: https://www.flecs.dev/explorer/?local=true&wasm=https://www.flecs.dev/explorer/playground.js https://www.flecs.dev/explorer/?local=true&wasm=https://www.... - and a lot more: https://www.flecs.dev/flecs/md_docs_Quickstart.html#autotoc_md33 https://www.flecs.dev/flecs/md_docs_Quickstart.html#autotoc_...
  - indigoabstract 4y ago
    Yes, I think I like the overall EnTT design, but I've only toyed around with the ECS part, to learn how to use it. I still haven't entirely figured out when it's best to use an ECS and when to use a scene graph and if it makes any sense to use both at the same time in a project.
    - 4y ago
korijn 4y ago
Any idea what a game loop would look like for an ECS engine that only renders a new frame when some animation is running or when the user is manipulating the camera, but not otherwise? Let's not argue "why". Most obvious would be saving battery.
- ajmmertens 4y ago
  Definitely possible. I know of a project that used an event-based architecture to decide when to run ECS systems.
- jiggawatts 4y ago
  I wrote a game engine that did that, because it was for a very simple, largely static scene. You would implement it almost the same as a standard Windows GUI -- using an event handler loop that marks the screen as "dirty" and needing a refresh if it has changed. Where it would immediately break down is ongoing animations. If you stopped moving your mouse, then animations would stop. Of course, you can set up the event loop to continue triggering if there are animations going on, but then you basically have a standard game engine loop running most of the time. Personally, I'd be happy if games simply paused rendering when minimised. Some games do this, some don't.
  - exDM69 4y ago
    > I'd be happy if games simply paused rendering when minimised. Some games do this, some don't. Yes, please. It drives me nuts to have GPU fans spinning when game is minimized, paused, in the loading screen, turn based and no inputs, etc.
  - korijn 4y ago
    Well, if those animations are things such as camera smoothing, it won't have such an impact. But I see your point. Thanks for entertaining the idea!
    - cnity 4y ago
      You just have to include all the parameters of the animation as part of your "dirtiness" calculation. I reckon in practice this means centralising pretty much _all game state_ in some way that you force the dirtiness state to be set correctly.
deleted 4y ago
[deleted]
holografix 4y ago
Could this be used to implement an efficient backend server for an Unreal Engine game?
- ajmmertens 4y ago
  It could be yep. One of the things that makes it a good fit for this use case is that Flecs has a query language that supports things like graph traversal and joins. Combined with either the builtin REST API or some custom protocol you could use it as a realtime in-memory database (which is what I use it for).
- Cthulhu_ 4y ago
  Doesn't Unreal have its own entity system yet?
  - devboi 4y ago
    It does and it is called Mass Entity.
  - illiarian 4y ago
    (not a game dev, don't quote me :) ) Unless I'm mistaken, Unreal hasn't had an official blessed ECS system relying on other things: from just having regular objects to reading data from data tables. They did introduce an ECS with (currently experimental) Mass Entity in Unreal 5: https://docs.unrealengine.com/5.0/en-US/mass-entity-in-unreal-engine/ https://docs.unrealengine.com/5.0/en-US/mass-entity-in-unrea...
- chii 4y ago
  i dont think it's tied to any particular engine or renderer, because an ECS is merely a mechanism to organize data so that it can be accessed fast.
deterministic 4y ago
Experienced game developer here: if you are just starting out wanting to learn how to make games then 1000000% focus on gameplay and not technology. Almost all developers I know who starts working on games end up basically just working on engine technology and never get an actual game done. I used to be one of them so I talk from experience! :) Nowadays I just use Unity and focus on the actual hardest part of making games: coming up with fun enjoyable gameplay that isn’t just a clone of somebody else’s game.
strictfp 4y ago
I'm working with an ECS at $dayjob. But I'm starting to question one aspect of them. Composition of functionality makes sense. And systems acting independently and only on objects with related components makes sense. But dynamically updating the set of components for an entity in order to "send it" to some other system, and relying on the ECS as a query database to coordinate all this, seems like questionable practice to me. I'm seeing more and more of that in our systems. I think ECSes are becoming too dynamic, and will start to eat perf, and hide relations. Like some message bus. Anyone else with the same gut feeling?
- vouwfietsman 4y ago
  > But dynamically updating the set of components for an entity in order to "send it" to some other system, and relying on the ECS as a query database to coordinate all this, seems like questionable practice to me. Indeed it is. The idea of components is to decouple systems. Where systems generally operate on few components, and own a disjoint set of components. It is not a communication channel, just like any other database should not (primarily) be used as a communication channel. You can combine an ECS with any number of different patterns to make your communication work: events, agents, messagebus, you name it.
- CapsAdmin 4y ago
  Apart from memory layout benefits, I always felt ECS is a bit like giving dynamic language capabilities to a statically typed language.
- worldsayshi 4y ago
  Interesting. Do you have any alternative approach in mind?
- Carbonhell 4y ago
  If I understood correctly, you are referring to applying some sort of tag component to a set of entity to be able to query it in a separate system. Am I correct? For cross system data flow, I think a mechanism similar to what Bevy does is the way to go. A specific event bus where the developer can send any type of struct, which can also allow passing a set of entity ids. I have briefly used it for my project and I think it's clear to reason about. The only catch is that you need to be careful about timing (the time between sending and receiving an event might take one frame to propagate). https://bevy-cheatbook.github.io/programming/events.html https://bevy-cheatbook.github.io/programming/events.html
grenoire 4y ago
To any budding game developers, my advice would be to not rush into ECS and try to accomplish your goals with structs and loops first and understand the fundamentals of how a video game 'works' under the hood. Your game will get done if you work on it, not necessarily if you pick ECS!
- brigadier132 4y ago
  I completely disagree. How games work is simple. It's a simulation run at some frequency and the simulation can be affected by player input. Sometimes there is a synchronization step with a server and that is more complicated. ECS is a pattern that results in better performance and more maintainable code. There is no reason not to use it.
  - deleted 4y ago
    [deleted]
  - HelloNurse 4y ago
    Better performance and more maintainable code are never the only concerns. For example, achieving a running prototype quickly with some ported/adapted non-ECS engine from a previous project might be more effective and efficient that starting from scratch with a better ECS architecture.
- whateveracct 4y ago
  In imperative langs, maybe. But I _would_ recommend ECS to Haskell devs interested in making games because apecs is just that good.
- pjmlp 4y ago
  Indeed, golden rules of game development, finish the game, game design matters more than tech, either make an engine or a game.
  - ajmmertens 4y ago
    The opposite also applies: don't reinvent the wheel if you want to get something done. Existing game engines like Unity/Unreal already have builtin solutions for entity management, but if you're (for whatever reason) using something that's more custom you can avoid having to reinvent entity management by using an ECS library. Similar to how you'd use libraries like SDL/bgfx/sokol for graphics.
markpapadakis 4y ago
https://github.com/skypjack/entt https://github.com/skypjack/entt is a fantastic alternative; also, one of the most beautiful codebases/elegant designs I 've come across.
stkni 4y ago
I have a sort of off-topic question. The marketing for this claims not using STL containers as a selling point. Does anyone know what makes STL containers unsuitable for this use case? Clearly they're not a one size fits all but I'm intrigued to know why you market an unknown container based API over a tried and tested one?
- ajmmertens 4y ago
  The biggest reason for me was to reduce compile times, which got cut into approximately half when I replaced STL containers with simpler custom ones. This starts adding up in large projects, where lots of files end up including the ECS.
- doodlesdev 4y ago
  Start here: https://youtu.be/6hC9IxqdDDw https://youtu.be/6hC9IxqdDDw

200 points by apitman 4y ago

I read the front page of this site, and I still have a general question about ECSes. For what it's worth, I'm developing a DAW (digital audio workstation) in Rust. It seems that ECS helps compose "things" (an entity) out of groups of data (each a component), and then operates upon combinations of those components (using systems). My domain is organized around behavior (many musical instruments implementing a Rust trait or interface, such as MIDI), but the data constituting each instrument is totally different. So the behaviors are the same, but the data is different. It seems that ECS isn't a fit, which is a shame because the implementations I've seen offer cool features like easy parallel processing, querying, and so on. Is there a design pattern like ECS but for common behaviors instead of common data? Or am I being dense and not seeing how ECS actually is a good fit for this situation?

ajmmertens 4y ago
The way you frame the question already suggests a solution, and one that indeed doesn't fit ECS that well. That doesn't mean you couldn't design a DAW using ECS, but you'd have to redesign how it works. For example: - create custom components for different instruments - create systems for specific instruments that match the necessary components - write the result to a common component (e.g. "ChannelOutput") You could then add components for EQ, effects, routing, ... and introduce systems that match those. The advantage of this approach would be that you end up with a bunch of loosely coupled systems (for instruments, effects, mixing, ...) and an in-memory representation that's cache & parallelization friendly. One other datapoint would be that Unreal's animation sequencer uses an ECS design, which has a lot of things in common with a DAW. Glossing over lots of details here, you can't of course design a DAW in one paragraph.
- sowbug 4y ago
  Thanks (and thanks especially for responding to what should have been a Stack Overflow question). I do see potential for certain ECS systems that span instruments, like DCA, effects wet/dry, routing, and mute. That makes sense. Your suggestion for component/system per instrument was what I felt ECS was asking me to do, but the libraries I investigated seemed to contemplate dozens but not hundreds of components, and that's where I began to think I was putting the cart before the horse. I had an epiphany a while ago that a modern interactive DAW is a video game, except parts of it run at 44,100 fps. And even the "except" part is misleading, because games have sound as well. That opened up my mind to game development as another domain to steal designs from.

indigoabstract 4y ago

This looks pretty well polished and maintained, I wasn't aware of it. So far, I've used an alternative ECS called entt: https://github.com/skypjack/entt https://github.com/skypjack/entt https://www.libhunt.com/compare-flecs-vs-entt?ref=compare https://www.libhunt.com/compare-flecs-vs-entt?ref=compare

pphysch 4y ago
EnTT is quite nice because it's more than just a ECS (if you want), it also includes libraries for a variety of other features like resource management. It's more of a "game engine" library, which is fantastic for projects that don't need to muck about in a "game studio" like Unity/Unreal (e.g. for games or simulations that don't require traditional level design).
- ajmmertens 4y ago
  The same goes for Flecs: - it has tools like https://github.com/flecs-hub/explorer https://github.com/flecs-hub/explorer that work with any Flecs project - it has builtin support for hierarchies/prefabs and more (https://ajmmertens.medium.com/building-games-in-ecs-with-entity-relationships-657275ba2c6c https://ajmmertens.medium.com/building-games-in-ecs-with-ent...) - it has optional support for systems/scheduling - it has an optional reflection framework - it has an optional DSL for describing assets/scenes, with an online playground to try it out: https://www.flecs.dev/explorer/?local=true&wasm=https://www.flecs.dev/explorer/playground.js https://www.flecs.dev/explorer/?local=true&wasm=https://www.... - and a lot more: https://www.flecs.dev/flecs/md_docs_Quickstart.html#autotoc_md33 https://www.flecs.dev/flecs/md_docs_Quickstart.html#autotoc_...
- indigoabstract 4y ago
  Yes, I think I like the overall EnTT design, but I've only toyed around with the ECS part, to learn how to use it. I still haven't entirely figured out when it's best to use an ECS and when to use a scene graph and if it makes any sense to use both at the same time in a project.
  - 4y ago

korijn 4y ago

Any idea what a game loop would look like for an ECS engine that only renders a new frame when some animation is running or when the user is manipulating the camera, but not otherwise? Let's not argue "why". Most obvious would be saving battery.

ajmmertens 4y ago
Definitely possible. I know of a project that used an event-based architecture to decide when to run ECS systems.
jiggawatts 4y ago
I wrote a game engine that did that, because it was for a very simple, largely static scene. You would implement it almost the same as a standard Windows GUI -- using an event handler loop that marks the screen as "dirty" and needing a refresh if it has changed. Where it would immediately break down is ongoing animations. If you stopped moving your mouse, then animations would stop. Of course, you can set up the event loop to continue triggering if there are animations going on, but then you basically have a standard game engine loop running most of the time. Personally, I'd be happy if games simply paused rendering when minimised. Some games do this, some don't.
- exDM69 4y ago
  > I'd be happy if games simply paused rendering when minimised. Some games do this, some don't. Yes, please. It drives me nuts to have GPU fans spinning when game is minimized, paused, in the loading screen, turn based and no inputs, etc.
- korijn 4y ago
  Well, if those animations are things such as camera smoothing, it won't have such an impact. But I see your point. Thanks for entertaining the idea!
  - cnity 4y ago
    You just have to include all the parameters of the animation as part of your "dirtiness" calculation. I reckon in practice this means centralising pretty much _all game state_ in some way that you force the dirtiness state to be set correctly.

deleted 4y ago

[deleted]

holografix 4y ago

Could this be used to implement an efficient backend server for an Unreal Engine game?

ajmmertens 4y ago
It could be yep. One of the things that makes it a good fit for this use case is that Flecs has a query language that supports things like graph traversal and joins. Combined with either the builtin REST API or some custom protocol you could use it as a realtime in-memory database (which is what I use it for).
Cthulhu_ 4y ago
Doesn't Unreal have its own entity system yet?
- devboi 4y ago
  It does and it is called Mass Entity.
- illiarian 4y ago
  (not a game dev, don't quote me :) ) Unless I'm mistaken, Unreal hasn't had an official blessed ECS system relying on other things: from just having regular objects to reading data from data tables. They did introduce an ECS with (currently experimental) Mass Entity in Unreal 5: https://docs.unrealengine.com/5.0/en-US/mass-entity-in-unreal-engine/ https://docs.unrealengine.com/5.0/en-US/mass-entity-in-unrea...
chii 4y ago
i dont think it's tied to any particular engine or renderer, because an ECS is merely a mechanism to organize data so that it can be accessed fast.

deterministic 4y ago

Experienced game developer here: if you are just starting out wanting to learn how to make games then 1000000% focus on gameplay and not technology. Almost all developers I know who starts working on games end up basically just working on engine technology and never get an actual game done. I used to be one of them so I talk from experience! :) Nowadays I just use Unity and focus on the actual hardest part of making games: coming up with fun enjoyable gameplay that isn’t just a clone of somebody else’s game.

strictfp 4y ago

I'm working with an ECS at $dayjob. But I'm starting to question one aspect of them. Composition of functionality makes sense. And systems acting independently and only on objects with related components makes sense. But dynamically updating the set of components for an entity in order to "send it" to some other system, and relying on the ECS as a query database to coordinate all this, seems like questionable practice to me. I'm seeing more and more of that in our systems. I think ECSes are becoming too dynamic, and will start to eat perf, and hide relations. Like some message bus. Anyone else with the same gut feeling?

vouwfietsman 4y ago
> But dynamically updating the set of components for an entity in order to "send it" to some other system, and relying on the ECS as a query database to coordinate all this, seems like questionable practice to me. Indeed it is. The idea of components is to decouple systems. Where systems generally operate on few components, and own a disjoint set of components. It is not a communication channel, just like any other database should not (primarily) be used as a communication channel. You can combine an ECS with any number of different patterns to make your communication work: events, agents, messagebus, you name it.
CapsAdmin 4y ago
Apart from memory layout benefits, I always felt ECS is a bit like giving dynamic language capabilities to a statically typed language.
worldsayshi 4y ago
Interesting. Do you have any alternative approach in mind?
Carbonhell 4y ago
If I understood correctly, you are referring to applying some sort of tag component to a set of entity to be able to query it in a separate system. Am I correct? For cross system data flow, I think a mechanism similar to what Bevy does is the way to go. A specific event bus where the developer can send any type of struct, which can also allow passing a set of entity ids. I have briefly used it for my project and I think it's clear to reason about. The only catch is that you need to be careful about timing (the time between sending and receiving an event might take one frame to propagate). https://bevy-cheatbook.github.io/programming/events.html https://bevy-cheatbook.github.io/programming/events.html

grenoire 4y ago

To any budding game developers, my advice would be to not rush into ECS and try to accomplish your goals with structs and loops first and understand the fundamentals of how a video game 'works' under the hood. Your game will get done if you work on it, not necessarily if you pick ECS!

brigadier132 4y ago
I completely disagree. How games work is simple. It's a simulation run at some frequency and the simulation can be affected by player input. Sometimes there is a synchronization step with a server and that is more complicated. ECS is a pattern that results in better performance and more maintainable code. There is no reason not to use it.
- deleted 4y ago
  [deleted]
- HelloNurse 4y ago
  Better performance and more maintainable code are never the only concerns. For example, achieving a running prototype quickly with some ported/adapted non-ECS engine from a previous project might be more effective and efficient that starting from scratch with a better ECS architecture.
whateveracct 4y ago
In imperative langs, maybe. But I _would_ recommend ECS to Haskell devs interested in making games because apecs is just that good.
pjmlp 4y ago
Indeed, golden rules of game development, finish the game, game design matters more than tech, either make an engine or a game.
- ajmmertens 4y ago
  The opposite also applies: don't reinvent the wheel if you want to get something done. Existing game engines like Unity/Unreal already have builtin solutions for entity management, but if you're (for whatever reason) using something that's more custom you can avoid having to reinvent entity management by using an ECS library. Similar to how you'd use libraries like SDL/bgfx/sokol for graphics.

markpapadakis 4y ago

https://github.com/skypjack/entt https://github.com/skypjack/entt is a fantastic alternative; also, one of the most beautiful codebases/elegant designs I 've come across.

stkni 4y ago

I have a sort of off-topic question. The marketing for this claims not using STL containers as a selling point. Does anyone know what makes STL containers unsuitable for this use case? Clearly they're not a one size fits all but I'm intrigued to know why you market an unknown container based API over a tried and tested one?

ajmmertens 4y ago
The biggest reason for me was to reduce compile times, which got cut into approximately half when I replaced STL containers with simpler custom ones. This starts adding up in large projects, where lots of files end up including the ECS.
doodlesdev 4y ago
Start here: https://youtu.be/6hC9IxqdDDw https://youtu.be/6hC9IxqdDDw