5 ms·

Essentially implementing JavaScript on top of the ECS architecture :D

by Permik 1y ago

aapoalas 1y ago
Yup! My whole inspiration for this came from a friend explaining ECS to me and me thinking "wouldn't that work for a JS engine?"
- SkiFire13 1y ago
  I've seen this brought up a couple times now, but I never get it. Why would ECS fit a JS engine? The ECS pattern optimizes for iterating over ton of data, but a JS engine does the opposite of that, it need to interpret instruction by instruction which could access random data.
  - aapoalas 1y ago
    Indeed, there's no guarantee that it will fit: I think it will but I don't know and want to find out. There are strong (IMO) reasons to think it will fit, though. User code can indeed do whatever but it rarely does. Programs written in JS are no less structured and predictable than ones written in C++ or Rust or any other language: they mostly operate on groups of data running iterations, loops, and algorithms over and over again. So the instructions being interpreted are likely to form roughly ECS System-like access patterns. Furthermore, it is more likely that data that came into the engine at one time (eg. one JSON.parse call or fetch result) will be iterated through at the same time. Thus, if the engine can ensure that data is and stays temporally colocated, then it is statistically likely that the interpreter's memory access patterns will not only come from System-like algorithms, they will access Component-array like memory. So: JS objects (and other heap allocated data) are Entities, their data is laid out in arrays of Components (TODO laying out object properties in Component arrays, at least in some cases), and the program forms the Systems. ECS :)
    - eyelidlessness 1y ago
      Disclaimer: I’m way out of my depth on the theoretical front, despite similarly taking interest in ECS in unconventional places. I’m responding from the perspective of most of my career being in JS/TS. I think your instincts about program structure are mostly right, but the outliers are pretty far out there. I’m much less optimistic about how you’re framing arbitrary data access. In my experience, it’s very common for JS code (marginally less common when authored as TS) to treat JSON (or other I/O bound data) as a perpetual blob of uncertainty. Data gets partially resolved into program interfaces haphazardly, at seemingly random points downstream, often with tons of redundancy and internal contradictions. I’m not sure how much that matters for your goals! But if I were taking on a project like this I’d be looking at that subset of non-ideal patterns frequently to reassess my assumptions.
      - aapoalas 1y ago
        Hey, thank you for the viewpoint. I'm myself a career JS/S programmer as well, and I do appreciate that the lived reality is quite varied. The partial resolving and haphazardness of JSON data usage shouldn't matter too much. I don't mean to make JSON parsed objects to be some special class, per se, or for the memory layout to depend on access patterns on said data. Only, I force data that was created together to be close together in memory (this is what real production engines already do, but only if possible) and for that data to stay together (again, production engines do this but only as is reasonably possible; I force the issue). So I explicitly choose temporal coherence. Beyond that, I use interface inheritance / removal of structural inheritance to reduce memory usage. eg. Plain Arrays (used in the common way) I can push to 9 bytes or even 8 bytes if I accept that Arrays with a length larger than 2^24 are always pessimised. ECS / Struct-of-Arrays data storage then further allows me to choose to move some data onto separate cache lines. But; it's definitely true that some programs will just ruin all reasonable access patterns and do everything willy-nilly and mixed up. I expect Nova to perform worse on those kinds of cases: as I am adding indirection to uncommon cases and splitting up data onto multiple cache lines to improve common access patterns, I do pessimise the uncommon cases further and further down the drain. I guess I just want to see what happens if I kick those uncommon cases to the curb and say "you want to be slow? feel free." :) I expect I will pay for that arrogance, and I look forward to that day <3
- chris37879 1y ago
  I'll be checking this project out! I'm a big fan of ECS and have lofty goals to use it for a data processing project I've been thinking about for a long time that has a lot in common with a programming language, enough that I've basically been considering it as one this whole time. So it's always cool to see ECS turn up somewhere I wouldn't otherwise expect it.

Essentially implementing JavaScript on top of the ECS architecture :D

by Permik 1y ago

Essentially implementing JavaScript on top of the ECS architecture :D

Yup! My whole inspiration for this came from a friend explaining ECS to me and me thinking "wouldn't that work for a JS engine?"

SkiFire13 1y ago
I've seen this brought up a couple times now, but I never get it. Why would ECS fit a JS engine? The ECS pattern optimizes for iterating over ton of data, but a JS engine does the opposite of that, it need to interpret instruction by instruction which could access random data.
- aapoalas 1y ago
  Indeed, there's no guarantee that it will fit: I think it will but I don't know and want to find out. There are strong (IMO) reasons to think it will fit, though. User code can indeed do whatever but it rarely does. Programs written in JS are no less structured and predictable than ones written in C++ or Rust or any other language: they mostly operate on groups of data running iterations, loops, and algorithms over and over again. So the instructions being interpreted are likely to form roughly ECS System-like access patterns. Furthermore, it is more likely that data that came into the engine at one time (eg. one JSON.parse call or fetch result) will be iterated through at the same time. Thus, if the engine can ensure that data is and stays temporally colocated, then it is statistically likely that the interpreter's memory access patterns will not only come from System-like algorithms, they will access Component-array like memory. So: JS objects (and other heap allocated data) are Entities, their data is laid out in arrays of Components (TODO laying out object properties in Component arrays, at least in some cases), and the program forms the Systems. ECS :)
  - eyelidlessness 1y ago
    Disclaimer: I’m way out of my depth on the theoretical front, despite similarly taking interest in ECS in unconventional places. I’m responding from the perspective of most of my career being in JS/TS. I think your instincts about program structure are mostly right, but the outliers are pretty far out there. I’m much less optimistic about how you’re framing arbitrary data access. In my experience, it’s very common for JS code (marginally less common when authored as TS) to treat JSON (or other I/O bound data) as a perpetual blob of uncertainty. Data gets partially resolved into program interfaces haphazardly, at seemingly random points downstream, often with tons of redundancy and internal contradictions. I’m not sure how much that matters for your goals! But if I were taking on a project like this I’d be looking at that subset of non-ideal patterns frequently to reassess my assumptions.
    - aapoalas 1y ago
      Hey, thank you for the viewpoint. I'm myself a career JS/S programmer as well, and I do appreciate that the lived reality is quite varied. The partial resolving and haphazardness of JSON data usage shouldn't matter too much. I don't mean to make JSON parsed objects to be some special class, per se, or for the memory layout to depend on access patterns on said data. Only, I force data that was created together to be close together in memory (this is what real production engines already do, but only if possible) and for that data to stay together (again, production engines do this but only as is reasonably possible; I force the issue). So I explicitly choose temporal coherence. Beyond that, I use interface inheritance / removal of structural inheritance to reduce memory usage. eg. Plain Arrays (used in the common way) I can push to 9 bytes or even 8 bytes if I accept that Arrays with a length larger than 2^24 are always pessimised. ECS / Struct-of-Arrays data storage then further allows me to choose to move some data onto separate cache lines. But; it's definitely true that some programs will just ruin all reasonable access patterns and do everything willy-nilly and mixed up. I expect Nova to perform worse on those kinds of cases: as I am adding indirection to uncommon cases and splitting up data onto multiple cache lines to improve common access patterns, I do pessimise the uncommon cases further and further down the drain. I guess I just want to see what happens if I kick those uncommon cases to the curb and say "you want to be slow? feel free." :) I expect I will pay for that arrogance, and I look forward to that day <3
chris37879 1y ago
I'll be checking this project out! I'm a big fan of ECS and have lofty goals to use it for a data processing project I've been thinking about for a long time that has a lot in common with a programming language, enough that I've basically been considering it as one this whole time. So it's always cool to see ECS turn up somewhere I wouldn't otherwise expect it.