3 ms·
It’s because it is built on top of the infrastructure that was introduce for Hack. The system first starts by allocating a large area in shared memory (through
by verlaguet 8y ago
It’s because it is built on top of the infrastructure that was introduce for Hack.
The system first starts by allocating a large area in shared memory (through a call to mmap). That area is very large but because most of it wont be written to it’s ok in practice.
After that, the program forks as many times as there are cores. Each of those cores are called “workers”. The first program is now the master.
The master and the workers communicate through pipes, but that is only used for synchronization. The lion share of the data goes through that shared memory that was mmaped at the beginning.
There are 2 main things shared in that area. 1- atomic hashtables of serialized ocaml objects (you can define as many as you like with a functor) 2- an atomic table of dependencies.
Each workers can read and write to those tables without locks (but only the master can remove from them).
It turns out that that setup works well in practice. Serializing/deserializing costs are mitigated by a cache of deserialized values for each core. And this way each core can manage there memory (the gc does not need to scan the shared memory).
Because that setup was working well in practice, it was reused by Flow and now Pyre. We refer to that setup internally as the “Hack infrastructure”.