10 ms·
Threads are very expensive if you start throwing C++ exceptions within them in parallel. You see the overall time to join the threads increases with each threa
by drmeister 7y ago
Threads are very expensive if you start throwing C++ exceptions within them in parallel. You see the overall time to join the threads increases with each thread you add. There is a mutex in the unwinding code and as the threads grab the mutex they invalidate each other's cache line. I wrote a demo to illustrate the problem https://github.com/clasp-developers/ctak https://github.com/clasp-developers/ctak
MacOS doesn't have this problem but Linux and FreeBSD do.
- wwarner 7y agoWow, I wouldn't have guessed that! Nice data!
- drmeister 7y agoThank you. We have developed a Common Lisp implementation that uses LLVM and interoperates with C++ and uses C++ exception handling to unwind the stack. Common Lisp code relies on stack unwinding a fair bit. Imagine my surprise when my fancy multi-threaded compiler can't get out of first gear (tops out at ~150% cpu) on Linux. Sheesh.
- gumby 7y agoI’ve been really interested in clasp; what’s the current state? Can’t see any recent posts that summarize it.
- drmeister 7y agoIt's going well. I'll post something soon. We've just been working on it quietly. We have multithreading, unicode, cffi etc, good debugging support, cross-language profiling and more.
- ajross 7y agoDid gcc/libstdc++ have the same problem? The report you link is for clang, it looks like.
- drmeister 7y agoYes, gcc/libstdc++ have the same problem. That's where I saw it first and then I tried llvm/libunwind and saw the same thing.
- monocasa 7y agoWhat's the actual lock on? The unwinding shouldn't involve shared mutable state (as someone who's been deep into the DWARF unwinding VM bytecode).
- drmeister 7y agoIn libgcc it's in Unwind_Find_FDE - we think it's a lock around walking the loaded dynamic libraries. I haven't personally dug much deeper into it but my folks here and the llvm engineers seem to be pretty certain that's the problem (this: https://github.com/gcc-mirror/gcc/blob/master/libgcc/unwind-dw2-fde.c#L1049 https://github.com/gcc-mirror/gcc/blob/master/libgcc/unwind-...). Right now we are rearranging our compiler so we throw fewer exceptions because you don't have to optimize things that you don't do :-).
- viraptor 7y agoLooks to me like it only tries to protect the building of the shared / sorted "seen_objects". You don't want two threads rebuilding it at the same time. Although there must be a way to work around this. Maybe something like optimistically walk through the seen list, then grab the lock to update and walk again without a lock? You should be able to safely walk a linked list forwards even with another thread inserting into it, right?
- gpderetta 7y agoThere is some work libc size to make the lock optional and enabling it only at the first dlopen. Edit: last time I investigated the issue ended up here: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=71744 https://gcc.gnu.org/bugzilla/show_bug.cgi?id=71744
- gumby 7y agoThere’s an easy optimization to avoid inspecting every frame when unwinding which c++ could not implement (for policy reasons) though a platform could: add a pointer to the next frame that needs unwinding to the frame setup. This is like move elision. If my caller has destructors to run or a catch clause this pointer is null and inspection proceeds as normal. If It does not it stores the value from its frame there. Then if I throw an exception I jump to the next frame that needs inspection; if I don’t then any throw further down the call stack won’t even look at me. The C++ standard can’t call for this because of the “zero cost if you don’t use it” rule. But a Linux ABI could. The MacOS takes advantage of this kind of freedom.
- brandmeyer 7y agoThis sounds a lot like the SJLJ runtime model that was used in G++ for years. > The C++ standard can’t call for this because of the “zero cost if you don’t use it” rule. But a Linux ABI could. The MacOS takes advantage of this kind of freedom. That's not really true. It has nothing to do with the standard. It has everything to do with the compiler's users complaining about the performance hit relative to DWARF EH. It is part of the social contract between the standards body, the compiler author community, and the user community that unused features don't cost us in runtime performance.
- gumby 7y agoYes, it's similar I suppose, though much lower overhead. I was a bigger advocate for frame inspection than Michael was in the early years because of my Lisp background. He was (correctly) more concerned with performance. As for "policy" vs "social contract" I think we basically agree.
- drmeister 7y agoThis sounds interesting - thank you! We need to interoperate with C++ - so we couldn't use this, could we? We could add this pointer to our own frames but C++ frames won't have this info - so I'm not sure how they would interoperate. We need to be able to throw an exception and invoke both Clasp frame cleanups and C++ frame cleanups up the stack. We have a crazy mix of C++ and CL frames on the stack at any time.
- lachlan-sneff 7y agoUnrelated to this thread, but are you still working on the matter compiler project?
- drmeister 7y agoThat's the point of everything. We have spun up a company and big things are in the works: http://www.thirdlaw.tech/ http://www.thirdlaw.tech/
- lachlan-sneff 7y agoAre you calling the software the "matter compiler" or is a "matter compiler"/"nanofactory" an eventual end goal of the project?
- ampdepolymerase 7y agoWhat about Windows?
- Koshkin 7y ago> start throwing C++ exceptions Well, don't. "Exception-based programming" is an anti-pattern. Exceptions should be thrown in, well, exceptional situations.
- monocasa 7y agoThey were using it in another language's runtime. Idiomatic advice like that doesn't always transfer.
- yjftsjthsd-h 7y agoAny idea why Darwin is beating Linux and FreeBSD here? Are they doing something different that could/should be implemented in the others?
- saagarjha 7y agoI believe Darwin has a special unwinding ABI that it uses.