3 ms·

>Adding proper mutexes to some of our code in production slowed things down so much it was actually better to just run the service under daemontools and let the

>Adding proper mutexes to some of our code in production slowed things down so much it was actually better to just run the service under daemontools and let the service crash/restart. Can the author explain this bit some more? If adding mutexes slows down your code to the point where you would rather just deal with data corruption/races it seems to me that there are issues with your code, not the language.

pcwalton 11y ago
Yeah, that seems weird. A properly implemented mutex is one atomic instruction in the uncontended path to lock and one more to unlock. And if you remove the mutex, you're turning "the contended path" into "the path that blows up your program". Atomic instructions can be painful on, for example, ARM (where they turn into load linked/store conditional loop + memory barrier) but they shouldn't be too bad on x86 servers. (I have no idea how Go implements mutexes, though.)
- seanmcdirmid 11y ago
  It is not unprecedented, e.g. see: http://en.wikipedia.org/wiki/Crash-only_software http://en.wikipedia.org/wiki/Crash-only_software Martin Rinard at MIT has done a lot of work in this area. It can be safe if you can rollback to a good state to avoid data corruption. Glitch, my current project, does not bother with reader locks for multi-core execution; instead it can rollback and retry when a write-after-read is detected effects are logged (dependencies are traced so write-after-reads can be detected). It has performance benefits if you can amortize the book keeping overhead, and we need to do it anyways to support live programming (see: http://research.microsoft.com/en-us/people/smcdirm/managedtime.aspx http://research.microsoft.com/en-us/people/smcdirm/managedti...).
  - nemothekid 11y ago
    I'm aware of crash only software (isn't this how Erlang/OTP does things?), however this doesn't seem to be what the OP intended. Running your code without caring about data races means your data can be corrupted and program might not crash.
    - seanmcdirmid 11y ago
      It actually depends on what the code does, we don't get much context on what the code is actually doing. It is completely possible that data corruption is not a concern because nothing is being written, or there are checksums or whatever...
    - vezzy-fnord 11y ago
      In this case, it's different. The Erlang/OTP approach involves maintaining internal supervisor trees that are bound to the semantics of the VM and its language constructs, whereas the author's involves using an OS process supervisor (daemontools) to just reexec the program every time it crashes/hangs. The potentials for data corruption and tainted state is higher in the latter, though it's still relatively good for a surprising number of problems.
- nly 11y ago
  I think the shitty ARM mutex problem you're describing was fixed in ARM8?

pcwalton 11y ago

Yeah, that seems weird. A properly implemented mutex is one atomic instruction in the uncontended path to lock and one more to unlock. And if you remove the mutex, you're turning "the contended path" into "the path that blows up your program". Atomic instructions can be painful on, for example, ARM (where they turn into load linked/store conditional loop + memory barrier) but they shouldn't be too bad on x86 servers. (I have no idea how Go implements mutexes, though.)

seanmcdirmid 11y ago
It is not unprecedented, e.g. see: http://en.wikipedia.org/wiki/Crash-only_software http://en.wikipedia.org/wiki/Crash-only_software Martin Rinard at MIT has done a lot of work in this area. It can be safe if you can rollback to a good state to avoid data corruption. Glitch, my current project, does not bother with reader locks for multi-core execution; instead it can rollback and retry when a write-after-read is detected effects are logged (dependencies are traced so write-after-reads can be detected). It has performance benefits if you can amortize the book keeping overhead, and we need to do it anyways to support live programming (see: http://research.microsoft.com/en-us/people/smcdirm/managedtime.aspx http://research.microsoft.com/en-us/people/smcdirm/managedti...).
- nemothekid 11y ago
  I'm aware of crash only software (isn't this how Erlang/OTP does things?), however this doesn't seem to be what the OP intended. Running your code without caring about data races means your data can be corrupted and program might not crash.
  - seanmcdirmid 11y ago
    It actually depends on what the code does, we don't get much context on what the code is actually doing. It is completely possible that data corruption is not a concern because nothing is being written, or there are checksums or whatever...
  - vezzy-fnord 11y ago
    In this case, it's different. The Erlang/OTP approach involves maintaining internal supervisor trees that are bound to the semantics of the VM and its language constructs, whereas the author's involves using an OS process supervisor (daemontools) to just reexec the program every time it crashes/hangs. The potentials for data corruption and tainted state is higher in the latter, though it's still relatively good for a surprising number of problems.
nly 11y ago
I think the shitty ARM mutex problem you're describing was fixed in ARM8?