3 ms·

Do you have any specific reason to say why it will cause miscompilation? Anything in particular that makes this UB? My understanding is that as long as your fun

by OskarS 4y ago

Do you have any specific reason to say why it will cause miscompilation? Anything in particular that makes this UB? My understanding is that as long as your functions compile to the correct ABI, it shouldn't be a problem. All you're doing is setting different stack/frame pointers before calling the function. As for the errno thing: the whole point about coroutines is that they are cooperatively scheduled, they don't yield unless you specifically tell them to. As long as you don't insert a yield-point in between the function that sets errno and then reading the value (and you should also always read errno immediately anyway), I don't see what the problem is (except errno being a terrible API in general, but that ship has pretty much sailed).

bonzini 4y ago
Yes, I do---we experienced it in QEMU and the workaround is hideous (https://patchew.org/QEMU/20211201170120.286139-1-stefanha@redhat.com/ https://patchew.org/QEMU/20211201170120.286139-1-stefanha@re...). What happens is that errno can be compiled to address_of_errno = &errno; ... foo = *address_of_errno; The compiler can assume that the address stays the same across the whole execution of the function and cache it in a register. Unfortunately, if you can change threads across yield points, you can end up reading the errno of the wrong function. Instead, C++ stackless coroutines explicitly promise that you will read the TLS of the currently executing thread in this case. This is what we found, but I cannot rule out that this is the only possibility.
- OskarS 4y ago
  That's really interesting, I hadn't considered the case of the compiler caching the address of errno. Obviously it can't cache the value across a function call (with or without a yield point), but caching the address does make sense, and I see how that would blow up if the coroutine was scheduled on a different thread. I'd argue that the problem here is that you should obviously not use TLS with corutines, since they can be scheduled on any thread. The concept of thread-local storage just doesn't make much sense for coroutines. The issue isn't necessarily that there's anything wrong with coroutines, it's rather with APIs that use TLS as an information side-channel. errno is just a bad idea, basically. But point taken.
  - bonzini 4y ago
    Using TLS in coroutines may not make much sense, but using TLS in the runtime can be unavoidable. For example, the stack of running coroutines is per-thread. You can perhaps avoid that with a more complex API, but placing it in TLS is the obvious design. Unfortunately, this has happened and is not hypothetical, but at least for now we're stuck in the stackful hole we dug for ourselves. :( A few years back we had similar issues on Windows.

by OskarS 4y ago

bonzini 4y ago

Yes, I do---we experienced it in QEMU and the workaround is hideous (https://patchew.org/QEMU/20211201170120.286139-1-stefanha@redhat.com/ https://patchew.org/QEMU/20211201170120.286139-1-stefanha@re...). What happens is that errno can be compiled to address_of_errno = &errno; ... foo = *address_of_errno; The compiler can assume that the address stays the same across the whole execution of the function and cache it in a register. Unfortunately, if you can change threads across yield points, you can end up reading the errno of the wrong function. Instead, C++ stackless coroutines explicitly promise that you will read the TLS of the currently executing thread in this case. This is what we found, but I cannot rule out that this is the only possibility.

OskarS 4y ago
That's really interesting, I hadn't considered the case of the compiler caching the address of errno. Obviously it can't cache the value across a function call (with or without a yield point), but caching the address does make sense, and I see how that would blow up if the coroutine was scheduled on a different thread. I'd argue that the problem here is that you should obviously not use TLS with corutines, since they can be scheduled on any thread. The concept of thread-local storage just doesn't make much sense for coroutines. The issue isn't necessarily that there's anything wrong with coroutines, it's rather with APIs that use TLS as an information side-channel. errno is just a bad idea, basically. But point taken.
- bonzini 4y ago
  Using TLS in coroutines may not make much sense, but using TLS in the runtime can be unavoidable. For example, the stack of running coroutines is per-thread. You can perhaps avoid that with a more complex API, but placing it in TLS is the obvious design. Unfortunately, this has happened and is not hypothetical, but at least for now we're stuck in the stackful hole we dug for ourselves. :( A few years back we had similar issues on Windows.