3 ms·

No, it doesn't. See "overcommit"

by l33tman 4y ago

enedil 4y ago
In a sense it does. Page tables would need to be copied.
- jacquesm 4y ago
  Yes, but they all point to the same pages. The tables take up a small fraction of the memory of the pages themselves.
  - enedil 4y ago
    But large fraction, if all you do afterwards is an exec call. Given 8 bytes per page table entry and 4k pages, it's 1/512 memory wasted. So if your process uses 8GB, it's 16MB. Still takes noticeable time if you spawn often.
    - jacquesm 4y ago
      I've never had the page tables be the cause of out of memory issues. Besides the fact that they are usually pre-allocated to avoid recursive page faults, but nothing would stop you from making the page tables themselves also copy-on-write during a fork.
    - lxgr 4y ago
      Aren't page tables nested? I don't know if any OS or hardware architecture actually supports it, but I could imagine the parent-level page table being virtual and copy-on-write itself.
reisse 4y ago
Please read the parent comments. Overcommit is necessary exactly because kernel has to reserve memory for both processes, and overcommit allows to reserve more memory than there is physically present. If kernel could not reserve memory for forked process, overcommit would not be necessary.
- lxgr 4y ago
  I think you've got it backwards: With overcommit, there is no memory reservation. The forked processes gets an exact copy of the other's page table, but with all writable memory marked as copy-on-write instead. The kernel might well be tallying these up to some number, but nothing important happens with it. Only without overcommit does the kernel does need to start accounting for hypothetically-writable memory before it actually is written to.
  - reisse 4y ago
    Yes, you're right here, that was what I wanted to say but probably was not able to formulate.
- blitzkrieg3 4y ago
  This is a misconception you and parent are perpetuating. fork() existed in this problematic 2x memory implementation _way_ before overcommit, and overcommit was non-existent or disabled on Unix (which has fork()) before Linux made it the default. Today with CoW we don't even have this "reserve memory for forked process" problem, so overcommit does nothing for us with regard to fork()/exec() (to say nothing of the vfork()/clone() point others have brought up). But if you want you can still disable overcommit on linux and observe that your apps can still create new processes. What overcommit enables is more efficient use of memory for applications that request more memory than they use (which is most of them) and more efficient use of page cache. It also pretty much guarantees an app gets memory when it asks for it, at the cost of getting oom-killed later if the system as a whole runs out.

No, it doesn't. See "overcommit"

enedil 4y ago

In a sense it does. Page tables would need to be copied.

jacquesm 4y ago
Yes, but they all point to the same pages. The tables take up a small fraction of the memory of the pages themselves.
- enedil 4y ago
  But large fraction, if all you do afterwards is an exec call. Given 8 bytes per page table entry and 4k pages, it's 1/512 memory wasted. So if your process uses 8GB, it's 16MB. Still takes noticeable time if you spawn often.
  - jacquesm 4y ago
    I've never had the page tables be the cause of out of memory issues. Besides the fact that they are usually pre-allocated to avoid recursive page faults, but nothing would stop you from making the page tables themselves also copy-on-write during a fork.
  - lxgr 4y ago
    Aren't page tables nested? I don't know if any OS or hardware architecture actually supports it, but I could imagine the parent-level page table being virtual and copy-on-write itself.

reisse 4y ago

Please read the parent comments. Overcommit is necessary exactly because kernel has to reserve memory for both processes, and overcommit allows to reserve more memory than there is physically present. If kernel could not reserve memory for forked process, overcommit would not be necessary.

lxgr 4y ago
I think you've got it backwards: With overcommit, there is no memory reservation. The forked processes gets an exact copy of the other's page table, but with all writable memory marked as copy-on-write instead. The kernel might well be tallying these up to some number, but nothing important happens with it. Only without overcommit does the kernel does need to start accounting for hypothetically-writable memory before it actually is written to.
- reisse 4y ago
  Yes, you're right here, that was what I wanted to say but probably was not able to formulate.
blitzkrieg3 4y ago
This is a misconception you and parent are perpetuating. fork() existed in this problematic 2x memory implementation _way_ before overcommit, and overcommit was non-existent or disabled on Unix (which has fork()) before Linux made it the default. Today with CoW we don't even have this "reserve memory for forked process" problem, so overcommit does nothing for us with regard to fork()/exec() (to say nothing of the vfork()/clone() point others have brought up). But if you want you can still disable overcommit on linux and observe that your apps can still create new processes. What overcommit enables is more efficient use of memory for applications that request more memory than they use (which is most of them) and more efficient use of page cache. It also pretty much guarantees an app gets memory when it asks for it, at the cost of getting oom-killed later if the system as a whole runs out.