4 ms·

Allocated-but-unavailable is a totally reasonable part of the memory hierarchy. Main Memory => zswap (compressed memory) => swap In this case, the pages may b

Allocated-but-unavailable is a totally reasonable part of the memory hierarchy. Main Memory => zswap (compressed memory) => swap In this case, the pages may be logically allocated or not -- the assurance is that the data will be the value you expect it to be when it becomes resident. Should those pages be uninitialized, the "Swapped" state is really just "Remember that this thing was all zeros." We could do computing your way, but it'd be phenomenally more expensive. I know this because every thing we introduce to the hierarchy in practice makes computing phenomenally less expensive.

cturner 5y ago
"We could do computing your way, but it'd be phenomenally more expensive" It must be viable - Windows prevents overcommit. But it has slow child-process-creation (edit: previously said "forking"), and this steers development towards native threads which is its own set of problems. I had never previously joined the dots on the point pcwalton makes at the top of this thread. It is a dramatic trade-off.
- pjc50 5y ago
  Windows does not have fork(), apart from the hidden ZwCreateProcess; https://news.ycombinator.com/item?id=9653975 https://news.ycombinator.com/item?id=9653975 For ages cygwin had to emulate it manually by hand-copying the process.
- throwawaylinux 5y ago
  Linux can prevent overcommit. And that's not the reason for much faster process creation than Windows as far as I know.
- Rusky 5y ago
  I'm not sure there's any causality between Windows preventing overcommit and Windows having slow process creation- there's nothing inherently slower about CreateProcess()/posix_spawn() than fork() + exec() as an API. It seems more that overcommit is a workaround for the way fork() can potentially lead to copying the entire address space into new pages, but usually doesn't. Because CreateProcess() knows more precisely how much to allocate before it returns to the child process, it can just reserve that amount and signal an error immediately if there's not enough memory+swap to back it. (And on the other hand, Windows has a lot of legacy and backwards compatibility behavior around processes that could easily explain the slower process creation independent of the API.)

cturner 5y ago

"We could do computing your way, but it'd be phenomenally more expensive" It must be viable - Windows prevents overcommit. But it has slow child-process-creation (edit: previously said "forking"), and this steers development towards native threads which is its own set of problems. I had never previously joined the dots on the point pcwalton makes at the top of this thread. It is a dramatic trade-off.

pjc50 5y ago
Windows does not have fork(), apart from the hidden ZwCreateProcess; https://news.ycombinator.com/item?id=9653975 https://news.ycombinator.com/item?id=9653975 For ages cygwin had to emulate it manually by hand-copying the process.
throwawaylinux 5y ago
Linux can prevent overcommit. And that's not the reason for much faster process creation than Windows as far as I know.
Rusky 5y ago
I'm not sure there's any causality between Windows preventing overcommit and Windows having slow process creation- there's nothing inherently slower about CreateProcess()/posix_spawn() than fork() + exec() as an API. It seems more that overcommit is a workaround for the way fork() can potentially lead to copying the entire address space into new pages, but usually doesn't. Because CreateProcess() knows more precisely how much to allocate before it returns to the child process, it can just reserve that amount and signal an error immediately if there's not enough memory+swap to back it. (And on the other hand, Windows has a lot of legacy and backwards compatibility behavior around processes that could easily explain the slower process creation independent of the API.)