3 ms·
I am making a new post because I feel like this is getting out of hand. There is a lot of misinformation that needs to be cleared up. When you are working with
by mbrumlow 10y ago
I am making a new post because I feel like this is getting out of hand. There is a lot of misinformation that needs to be cleared up.
When you are working with system calls you use fds to tell the kernel what file you are working with. These are simply numbers, an int to be exact. When you call a system call that takes in a fd you pass the int that references the file you want to work with and it is looked up in a table to find the kernel structure that represents the file. When you dup a fd all that does is add a entry to the lookup table to point to the same kernel structure -- increasing the refcount at the same time. The ref count lives on the kernel structure, not with the int. The fd system is real simple. For open, dupe, dup2… will add an entry. While close will remove the entry. If the entry is removed the fd is now useless for any communication to the kernel over system calls. This is because the lookup table will not be able to translate your fd to a kernel structure.
Now the question is epoll broken or have a bug? I don’t think so. You just have to keep in mind you are using a system call and not a library. This is controlled access to kernel level functions.
In the example adding a fd to epoll, then duping it followed by a closing is providing some surprising results. Understanding how userspace talks to kernel space and how system calls translate into running kernel code is key to understanding why this is not a bug but just how things work. You might also be surprised that other system calls behave the same way!
When you tell epoll to wait you use a fd to tell it which file you want to wait on. You are not telling a library, or a userspace program to do this. You are telling the kernel. So the fd is translated into a kernel object and that is what epoll is working with. Only when adding and removing is this translation done. The kernel subsystem that provides epoll will continue to work with kernel objects -- not fds.
When you call dup you are adding a new entry to your programs fd lookup table for a new fd but pointing to the same kernel structure -- at the same time you are also increment the refcount of the kernel structure. The file is not actually closed until both the original fd and the duped fd have had closed called on it, or more correctly, until the refcount of the kernel structure is zero. This means that if you tell epoll to wait on a file and do so by an fd, and then close the fd -- you will only see a close event if the refcount is zero. By calling dup before closing your original fd you have incremented the ref count and thus the file remains open -- no close event.
If you then try to remove your file from epoll using the now invalid (closed) fd it will fail. This is because the fd is no longer in the fd lookup table and the system call can not translate your fd into a kernel object.
Now you might say this is not how other system calls work. As in another thread pointed out that “read” does not behave this way. To the contrary -- read actually works the same way. If you dup your socket fd, and use a thread to close your socket fd while in an active read -- your read will not fail. It will finish out delivering the data to you. This is because the fd is not the actual object you are working with. It is just a reference to the object. Onces you enter the read function you are in kernel land. And the kernel only knows about the kernel object. Because you duped the fd before closing it the kernel object has a ref count of 2 and your close while read leaves the ref count at 1 thus there is nothing for the kernel to do other than finish out its read. If you attempt to use the now closed and invalid fd for a subsequent read, or close you will get a EBADF error -- much like you do when you attempt to use it with epolls functions to remove the file.
- Asooka 10y agoBut no other syscall interface presumes that you're directly twiddling kernel structs and I struggle to think of any other syscall that operates on fds, for which you need to keep in mind that it actually operates on internal kernel structs. It would be trivial for the kernel to attach the epoll struct to the file descriptor and give you a close notification when the descriptor is closed (even if there are other open descriptors to the same file), but it's really hard to emulate that behaviour in userland, except this is the precise behaviour most people want! The only way to emulate this behaviour is to write a full abstraction over all file syscalls and make sure you never ever call them directly.
- mbrumlow 10y agoThe entire point of my post was to say yes, that all the other/all system calls do work with kernel structs internally ones the system call passes into kernel land. Again, please do not forget a system call is a controlled exposure to kernel code -- it is not a library. Every function call works the same way as epoll works ounces the fd is translated into a actual kernel structure. Conflating the fd with the file is where all assumptions go wrong. The fd is the means you tell epoll what file you want to be notified on events about. A file is NOT closed until there are no more references. This is simply how linux works. To change that would be non trivial and break years of software. Let's explore your idea a bit more. If you were to want to know when a "fd" was closed, what would you be using that for? You really want to know when a file was closed right? If so, any code that you execute after you get the close on the fd would be wrong and cause even more surprising results. This is because the file is not actually closed. It is open. Whoever has the remaining handles can modify it while you think the file is closed -- that is a bug.