4 ms·

It's important to remember two things: First, in a more general way, fork() is kind of slow just by virtue of being a system call and involving several steps, s

It's important to remember two things: First, in a more general way, fork() is kind of slow just by virtue of being a system call and involving several steps, so applications try to avoid making lots of calls to fork (e.g. webservers long ago stopped doing the naïve fork-for-every-request model). Redis uses fork in a way nobody else seems to. Applications using tens of gigabytes of RAM -- databases, media editing, etc. -- just don't usually fork except during a startup process. Redis uses fork for persistence. It's clever, and it works great in general, but it's weird! Unique, even. It seems much more likely to hit a fork() weakness in a noticeable way than almost any other application out there.

antirez 12y ago
Totally true. I always think that eventually some kernel change may totally break how Redis works and I'll have to resort to implementing the same stuff in user space. A simpler parallel path is to use Redis Cluster / Sentinel and provide good safety for when persistence is disabled, and achieve data safety via redundancy in replicas. However even in such a scenario, it will be kind of handy that Redis can persist so that restarts performed with SHUTDOWN (for example for upgrades) will still retain the dataset when needed.
JoeAltmaier 12y ago
Fork has always been a mutant feature. Split the entire process into twins? So that the caller can immediately exec() and discard the tediously-cloned twin and become a different program? Its the most egregiously inefficient feature ever to grace an operating system. Page tables, stacks, heap allocations, file handles - all cloned and then discarded.
- antirez 12y ago
  In this Regard, Redis is the first system using it the proper way I guess ;-) I mean, all this copying is not discarded but used to create a point-in-time snapshot.
  - mcguire 12y ago
    No, every Unix command shell uses the copied program and file descriptors.
- dllthomas 12y ago
  Though these days it's worth noting that much of that is COW, cutting down the actual copying (at the expense of some bookkeeping). There's certainly still needless work, however.

antirez 12y ago

Totally true. I always think that eventually some kernel change may totally break how Redis works and I'll have to resort to implementing the same stuff in user space. A simpler parallel path is to use Redis Cluster / Sentinel and provide good safety for when persistence is disabled, and achieve data safety via redundancy in replicas. However even in such a scenario, it will be kind of handy that Redis can persist so that restarts performed with SHUTDOWN (for example for upgrades) will still retain the dataset when needed.

JoeAltmaier 12y ago

Fork has always been a mutant feature. Split the entire process into twins? So that the caller can immediately exec() and discard the tediously-cloned twin and become a different program? Its the most egregiously inefficient feature ever to grace an operating system. Page tables, stacks, heap allocations, file handles - all cloned and then discarded.

antirez 12y ago
In this Regard, Redis is the first system using it the proper way I guess ;-) I mean, all this copying is not discarded but used to create a point-in-time snapshot.
- mcguire 12y ago
  No, every Unix command shell uses the copied program and file descriptors.
dllthomas 12y ago
Though these days it's worth noting that much of that is COW, cutting down the actual copying (at the expense of some bookkeeping). There's certainly still needless work, however.