3 ms·

Why is running the DNS resolution thread a problem? It should be dequeuing resolution requests and pushing responses and sleeping when there is nothing to do W

by nly 1y ago

Why is running the DNS resolution thread a problem? It should be dequeuing resolution requests and pushing responses and sleeping when there is nothing to do When someone kills off the curl context surely you simply set a suicide flag on the thread and wake it up so it can be joined.

rwmj 1y ago
One problem may be that fork() kills background threads, so now any program that uses libcurl + fork has to have a new API to restart the DNS thread (or use posix_atfork which is a big PITA), and that might break existing programs using curl.
- loeg 1y ago
  A surmountable problem, sure.
  - rwmj 1y ago
    Sometimes. To give one counterexample, golang doesn't have a way to restart the threads it uses for I/O (apparently a decision the golang developers made), so if you're embedding golang code in another binary, it better not call fork. (The reason for this warning: https://gitlab.com/nbdkit/nbdkit/-/commit/2589e6da40939af9aea16a488e8f2aec411f3c5c https://gitlab.com/nbdkit/nbdkit/-/commit/2589e6da40939af9ae...)
    - loeg 1y ago
      This is a policy choice of Golang, but a C library like Curl (the topic of this thread) is not constrained by the policy choices of the Go developers. Curl could use MADV_WIPEONFORK or other primitives to restart its DNS thread automatically.
- ComputerGuru 1y ago
  It’s not too much of an exaggeration to say that everything about using fork() instead of vfork() plus exec() is essentially fundamentally broken in modern osdev without a whole stack of hacks to try and patch individual issues one-by-one.
  - EPWN3D 1y ago
    It's not an exaggeration in any sense. fork(2) basically cannot be done correctly in modern userspace stacks.
foota 1y ago
The thread started sounds like it's single use, not a thread handling requests in a loop. Anyway, a single thread handling requests in a loop would serialize these DNS lookups which if they're hanging would be problematic.
- loeg 1y ago
  Yes, but why? As GP notes, the thread doesn't have to be single-use.

by nly 1y ago

rwmj 1y ago

One problem may be that fork() kills background threads, so now any program that uses libcurl + fork has to have a new API to restart the DNS thread (or use posix_atfork which is a big PITA), and that might break existing programs using curl.

loeg 1y ago
A surmountable problem, sure.
- rwmj 1y ago
  Sometimes. To give one counterexample, golang doesn't have a way to restart the threads it uses for I/O (apparently a decision the golang developers made), so if you're embedding golang code in another binary, it better not call fork. (The reason for this warning: https://gitlab.com/nbdkit/nbdkit/-/commit/2589e6da40939af9aea16a488e8f2aec411f3c5c https://gitlab.com/nbdkit/nbdkit/-/commit/2589e6da40939af9ae...)
  - loeg 1y ago
    This is a policy choice of Golang, but a C library like Curl (the topic of this thread) is not constrained by the policy choices of the Go developers. Curl could use MADV_WIPEONFORK or other primitives to restart its DNS thread automatically.
ComputerGuru 1y ago
It’s not too much of an exaggeration to say that everything about using fork() instead of vfork() plus exec() is essentially fundamentally broken in modern osdev without a whole stack of hacks to try and patch individual issues one-by-one.
- EPWN3D 1y ago
  It's not an exaggeration in any sense. fork(2) basically cannot be done correctly in modern userspace stacks.

foota 1y ago

The thread started sounds like it's single use, not a thread handling requests in a loop. Anyway, a single thread handling requests in a loop would serialize these DNS lookups which if they're hanging would be problematic.

loeg 1y ago
Yes, but why? As GP notes, the thread doesn't have to be single-use.