3 ms·

> While it's true that user threads can't reset their signal masks without a system call, 1) SA_NODEFER means we don't have to, and 2) in theory, we could make

> While it's true that user threads can't reset their signal masks without a system call, 1) SA_NODEFER means we don't have to, and 2) in theory, we could make an rseq field that would allow us to implement sigprocmask(2) without an entry into the kernel! Writing a signal handler without SA_NODEFER is ... not fun. > I think you're right, and that's unfortunate. I'm still not sure, however, whether the need to support BPF stack-map insertion is sufficient to kill the userspace-driven unwinding scheme I've described, because (at least in my experience) this BPF-map-key scenario is a pretty fringe case. I don't think it's all that fringe, but yes, it's not as common as plain profiling. The thing that makes it the most niche is the difficulty around lack of frame pointers in libc - other libraries matter far less. > If I'm right about this impossibility, then we have to give up on one of 1) fully robust full (user+kernel) stack walking, and 2) stack-keyed eBPF maps. Which would you prefer? I'd give up 1). The stack page fault issue isn't all that big a deal in practice, although I do agree that it's quite ugly. Whereas being able to build things like "offwake" profiles (showing the stack of the waiter and the wakee + duration of sleeps) allows to debug/profile issues that are extremely hard to debug otherwise. And rely quite fundamentally on stack-keyed BPF maps.

quotemstr 4y ago
You don't need eBPF maps for wake-up profiles. If you just log wakeups the usual way, postprocessing tools get all the information they need to figure out what woke what. That people use eBPF maps for this purpose is a consequence of the inadequacy of Linux profiling tools.
- anarazel 4y ago
  IME that doesn't work, because the un-aggregated data volumes are far too high. The whole point of using something like eBPF is that you don't need to log all that data.
  - quotemstr 4y ago
    Is data volume too high? I don't see it. Windows and Android both collect useful traces and do post-hoc analysis on them instead of using things like eBPF stack maps to do online analysis.
    - anarazel 4y ago
      Android doesn't run highly concurrent workloads, so it's not surprising that you'd not fall over. But I haven't used it, so I really can't say anything well qualified. I've only used ETW a little, but I've found it to very quickly be overwhelmed with the number of events, even on quite small machines (vm with 4 cores). I think part of our different experience comes from using profiling and tracing in different workloads. I typically profile/trace while hacking on postgres, or occasionally when debugging production issues. Commonly very busy, many core workloads. If you're interested I can collect some numbers to show the difference between collecting all the trace events for an offwake trace, and using a map. From memory, using maps reduces throughout 20-30%, full collection way way more.

quotemstr 4y ago

You don't need eBPF maps for wake-up profiles. If you just log wakeups the usual way, postprocessing tools get all the information they need to figure out what woke what. That people use eBPF maps for this purpose is a consequence of the inadequacy of Linux profiling tools.

anarazel 4y ago
IME that doesn't work, because the un-aggregated data volumes are far too high. The whole point of using something like eBPF is that you don't need to log all that data.
- quotemstr 4y ago
  Is data volume too high? I don't see it. Windows and Android both collect useful traces and do post-hoc analysis on them instead of using things like eBPF stack maps to do online analysis.
  - anarazel 4y ago
    Android doesn't run highly concurrent workloads, so it's not surprising that you'd not fall over. But I haven't used it, so I really can't say anything well qualified. I've only used ETW a little, but I've found it to very quickly be overwhelmed with the number of events, even on quite small machines (vm with 4 cores). I think part of our different experience comes from using profiling and tracing in different workloads. I typically profile/trace while hacking on postgres, or occasionally when debugging production issues. Commonly very busy, many core workloads. If you're interested I can collect some numbers to show the difference between collecting all the trace events for an offwake trace, and using a map. From memory, using maps reduces throughout 20-30%, full collection way way more.