3 ms·

glibc's use of Intel TSX was also a bit of a tirefire, IMO. It got turned on by default for all locks, broke quite a few of applications (which were using lock

by rrss 7y ago

glibc's use of Intel TSX was also a bit of a tirefire, IMO. It got turned on by default for all locks, broke quite a few of applications (which were using locks wrong, but still), then the instructions ended up being broken in every Intel architecture to date that has ever implemented them [1]. Most of the distros put in some patches to disable the use of TSX, but the 'no-lock-elision' flag provided by glibc didn't actually disable all attempts at lock elision. So the end result is that the glibc in Ubuntu 16.04 tries to use TSX for rwlocks, but not mutexes. Which is unfortunate, because the mechanism to opt-out of the use of TSX in glibc was only implemented for mutexes, but not rwlocks. TSX only ever provided performance benefits in specific use cases, and at some point glibc upstream became sane and turned it all off by default and provided opt-in mechanisms instead. [1]: skylake errata: "Using Intel TSX Instructions May Lead to Unpredictable System Behavior", https://www.intel.com/content/dam/www/public/us/en/documents/specification-updates/xeon-scalable-spec-update.pdf https://www.intel.com/content/dam/www/public/us/en/documents...

saagarjha 7y ago
TSX is supposed to fall back to an alternative implementation anyways if there's a transactional abort, right? What's the issue with keeping it in?
- rrss 7y ago
  1. TSX has been broken in varying degrees in every architecture since haswell, and not always in a nice way like "always abort." In various steppings of haswell and early broadwell TSX was in various states of disablement, meaning that sometimes the instructions were advertised (in cpuid etc) to work but had subtle bugs, sometimes were advertised to work but were disabled such that any RTM instruction caused SIGILL, and sometimes correctly advertised to not work. The detection of the state of TSX was sufficiently unreliable that glibc ended up with a blacklist of CPU model numbers that were too problematic to even try using TSX. 2. The pthread implementation of mutex using TSX behaved differently from the prior implementation in cases of misuse. This was deemed acceptable because applications that were impacted were using mutexes illegally, but the result was that a ton of software in all the distros started crashing randomly (IIRC the scenario was a double unlock. pre-TSX glibc didn't complain, post-TSX the program crashed or something). 3. TSX was not generally beneficial. The glibc implementation did something like try using a transaction 3 or 5 times before giving up. The worst-case performance of starting the transaction, going as far into the critical section as possible, aborting, repeat 2x, then finally taking the lock normally is much worse than just taking the lock.

by rrss 7y ago

saagarjha 7y ago

TSX is supposed to fall back to an alternative implementation anyways if there's a transactional abort, right? What's the issue with keeping it in?

rrss 7y ago
1. TSX has been broken in varying degrees in every architecture since haswell, and not always in a nice way like "always abort." In various steppings of haswell and early broadwell TSX was in various states of disablement, meaning that sometimes the instructions were advertised (in cpuid etc) to work but had subtle bugs, sometimes were advertised to work but were disabled such that any RTM instruction caused SIGILL, and sometimes correctly advertised to not work. The detection of the state of TSX was sufficiently unreliable that glibc ended up with a blacklist of CPU model numbers that were too problematic to even try using TSX. 2. The pthread implementation of mutex using TSX behaved differently from the prior implementation in cases of misuse. This was deemed acceptable because applications that were impacted were using mutexes illegally, but the result was that a ton of software in all the distros started crashing randomly (IIRC the scenario was a double unlock. pre-TSX glibc didn't complain, post-TSX the program crashed or something). 3. TSX was not generally beneficial. The glibc implementation did something like try using a transaction 3 or 5 times before giving up. The worst-case performance of starting the transaction, going as far into the critical section as possible, aborting, repeat 2x, then finally taking the lock normally is much worse than just taking the lock.