4 ms·

It’s been decades, why doesn’t getaddrinfo have a standardised way to specify a timeout? Set a timeout to 10 seconds and life becomes a lot easier. Yes I know

It’s been decades, why doesn’t getaddrinfo have a standardised way to specify a timeout? Set a timeout to 10 seconds and life becomes a lot easier. Yes I know in Linux you can set the timeout in a config file. But really the dns setting should be configurable by the calling code. Some code requires fast lookups and doesn’t mind failing which, while others won’t mind waiting longer. It’s not a one size fits all thing.

o11c 1y ago
On Linux you can do what you're asking with `getaddrinfo_a` + `gai_suspend`. As always, on non-Linux Unixen the answer is "screw you!"
- surajrmal 1y ago
  The fact that a library as polific as libcurl doesn't use it and tried to resort to pthread_cancel, known to have serious problems, points to a larger problem. Either posix needs to incorporate an async getaddrinfo or there is an education problem that needs to be addressed.
- sedatk 1y ago
  Just wanted to note that Windows doesn't have that problem either. Even Windows NT had async getaddrinfo() variants.
- BobbyTables2 1y ago
  Wow, TIL ! Thanks!
- throwawayoogux 1y ago
  OpenBSD has getaddrinfo_async since 5.6 (March 2014).
ComputerGuru 1y ago
I disagree, there are too many variables and ultimately the end user would be th one that knows best. The proper solution isn’t having the library or application dev, who has no idea what kind of network connection the user is running, the type of dns server (caching or not, lan or remote, etc) or the name servers of the target domain and their performance or availability. This is all really the domain of the sysadmin. The solution is to make it a properly non-blocking api.
okanat 1y ago
Just leave DNS out, are there any POSIX standard async functionality for networking or even normal IO? All I know by reading some libraries is epoll or io_uring used on Linux, kevent on BSDs.
- mort96 1y ago
  POSIX has poll for that.
- comex 1y ago
  Yes for networking. You set your sockets into O_NONBLOCK mode and use poll() or select(). These APIs are in POSIX and also have direct equivalents in Winsock. There is also POSIX AIO for async I/O on any file descriptor, but at least historically speaking it doesn't work properly on Linux.
  - epcoa 1y ago
    POSIX AIO on “Linux” is implemented in glibc with userland threads using regular blocking syscalls behind the scenes. It basically works properly, it just doesn’t gain any potential efficiency benefits, it adds avoidable overhead, prone to priority inversion, etc. The linux kernel has no provision for POSIX AIO. Until io_uring the only asynchronous disk IO interface was the io_* syscalls, which were confusingly referred to as Asynchronous IO, though these have nothing to do with POSIX AIO and can only be used bypassing the page cache, and suck for general purpose use.
eklitzke 1y ago
A few reasons, I think. The first is that getaddrinfo is specified by POSIX, and the POSIX evolve very conservatively and at a glacial pace. The second reason is that specifying a timeout breaks symmetry with a lot of other functions in Unix/C, both system calls and libc calls. For example, you can't specify a timeout when opening a file, reading from a file, or closing a file, which are all potentially blocking operations. There are ways to do these things in a non-blocking manner with timeouts using aio or io_uring, but those are already relatively complicated APIs for just using simple system calls, and getaddrinfo is much more complicated. The last reason is that if you use the sockets APIs directly it's not that hard to write a non-blocking DNS resolver (c-ares is one example). The thing is though that if you write your own resolver you have to consider how to do caching, it won't work with NSS on Linux, etc. You can implement these things (systemd-resolved does it, and works with NSS) but they are a lot of work to do properly.
- jstimpfle 1y ago
  > For example, you can't specify a timeout when opening a file, reading from a file, or closing a file, which are all potentially blocking operations. No they're not. Not really, unless you consider disk access and interacting with the page cache/inode cache inside the kernel to be blocking. But if you do that, you should probably also consider scheduling and really any CPU instruction to be blocking. (If the system is too loaded, anything can be slow). To be fair, network requests can be considered non-blocking in a similar way, but they're depending on other systems that you generally can't control or inspect. In practice you'll see network timeouts. Note that you (at least normally -- there might be tricky exceptions) won't see EINTR from read() to a filesystem file. But you can see EINTR for network sockets. The difference is that, in Unix terminology, disks are not considered "slow devices".
  - surajrmal 1y ago
    In a data center, networks can have lower latency than disk (even ssd). Now the real place this all falls on its head is page faults. That there are definitely places where you need to have granular control over what can and cannot stall a thread from making progress.
asveikau 1y ago
I think getaddrinfo_a is cancellable, including the ability to block with a timeout. It is a glibc extension.

o11c 1y ago

On Linux you can do what you're asking with `getaddrinfo_a` + `gai_suspend`. As always, on non-Linux Unixen the answer is "screw you!"

surajrmal 1y ago
The fact that a library as polific as libcurl doesn't use it and tried to resort to pthread_cancel, known to have serious problems, points to a larger problem. Either posix needs to incorporate an async getaddrinfo or there is an education problem that needs to be addressed.
sedatk 1y ago
Just wanted to note that Windows doesn't have that problem either. Even Windows NT had async getaddrinfo() variants.
BobbyTables2 1y ago
Wow, TIL ! Thanks!
throwawayoogux 1y ago
OpenBSD has getaddrinfo_async since 5.6 (March 2014).

ComputerGuru 1y ago

I disagree, there are too many variables and ultimately the end user would be th one that knows best. The proper solution isn’t having the library or application dev, who has no idea what kind of network connection the user is running, the type of dns server (caching or not, lan or remote, etc) or the name servers of the target domain and their performance or availability. This is all really the domain of the sysadmin. The solution is to make it a properly non-blocking api.

okanat 1y ago

Just leave DNS out, are there any POSIX standard async functionality for networking or even normal IO? All I know by reading some libraries is epoll or io_uring used on Linux, kevent on BSDs.

mort96 1y ago
POSIX has poll for that.
comex 1y ago
Yes for networking. You set your sockets into O_NONBLOCK mode and use poll() or select(). These APIs are in POSIX and also have direct equivalents in Winsock. There is also POSIX AIO for async I/O on any file descriptor, but at least historically speaking it doesn't work properly on Linux.
- epcoa 1y ago
  POSIX AIO on “Linux” is implemented in glibc with userland threads using regular blocking syscalls behind the scenes. It basically works properly, it just doesn’t gain any potential efficiency benefits, it adds avoidable overhead, prone to priority inversion, etc. The linux kernel has no provision for POSIX AIO. Until io_uring the only asynchronous disk IO interface was the io_* syscalls, which were confusingly referred to as Asynchronous IO, though these have nothing to do with POSIX AIO and can only be used bypassing the page cache, and suck for general purpose use.

eklitzke 1y ago

A few reasons, I think. The first is that getaddrinfo is specified by POSIX, and the POSIX evolve very conservatively and at a glacial pace. The second reason is that specifying a timeout breaks symmetry with a lot of other functions in Unix/C, both system calls and libc calls. For example, you can't specify a timeout when opening a file, reading from a file, or closing a file, which are all potentially blocking operations. There are ways to do these things in a non-blocking manner with timeouts using aio or io_uring, but those are already relatively complicated APIs for just using simple system calls, and getaddrinfo is much more complicated. The last reason is that if you use the sockets APIs directly it's not that hard to write a non-blocking DNS resolver (c-ares is one example). The thing is though that if you write your own resolver you have to consider how to do caching, it won't work with NSS on Linux, etc. You can implement these things (systemd-resolved does it, and works with NSS) but they are a lot of work to do properly.

jstimpfle 1y ago
> For example, you can't specify a timeout when opening a file, reading from a file, or closing a file, which are all potentially blocking operations. No they're not. Not really, unless you consider disk access and interacting with the page cache/inode cache inside the kernel to be blocking. But if you do that, you should probably also consider scheduling and really any CPU instruction to be blocking. (If the system is too loaded, anything can be slow). To be fair, network requests can be considered non-blocking in a similar way, but they're depending on other systems that you generally can't control or inspect. In practice you'll see network timeouts. Note that you (at least normally -- there might be tricky exceptions) won't see EINTR from read() to a filesystem file. But you can see EINTR for network sockets. The difference is that, in Unix terminology, disks are not considered "slow devices".
- surajrmal 1y ago
  In a data center, networks can have lower latency than disk (even ssd). Now the real place this all falls on its head is page faults. That there are definitely places where you need to have granular control over what can and cannot stall a thread from making progress.

asveikau 1y ago

I think getaddrinfo_a is cancellable, including the ability to block with a timeout. It is a glibc extension.