3 ms·

It should kind of jump out for not being an "actually do something useful" syscall. Without knowing what your program does, what your syscalls say to me are:

by o11c 2y ago

It should kind of jump out for not being an "actually do something useful" syscall. Without knowing what your program does, what your syscalls say to me are: * why are so many files opened but not closed? * that's probably too much epoll_ctl; try optimistically performing the real syscall once before actually registering. Or, under some circumstances, do a quick few-fd `poll` since it is stateless. * is mmap really a win here? Unless you need its swap-like behavior, read is often faster (and it avoids ToCToU). Also, why are there fewer `fstat`s than `munmap`s? * why is there an ioctl so prominent; what is it even doing? * that's a lot of reading but no writing ... I guess maybe it's split across multiple lower syscalls but it's still weird. (of course, all these can have legitimate answers, but you need to know them)

spc476 2y ago
I do know how to answer most of the questions. The program is a gopher server. The list you see is the result of sending several hundred (sequential) requests to the server. The number of calls to open() vs. close() can be explained [1], so (to me) nothing unusual there. I don't directly call mmap(), munmap() or fstat(), so I'm assuming there's some code in some library I'm using that's doing those calls. And that's the assumption I would have with ioctl() as well (well, I now know better, but in investigating the issue originally I might just ignored ioctl() along with mmap()). There is a lot of reading, due to the requests (blog entries). Writing is to a socket. You are right in that there is too much epoll_ctl(), but it's unclear to me if I would have found the actual issue. I did do change the code to perform the real system call before registering, but that had little effect. [1] Not all requests are valid, for instance. In this case, it's a mirror of my blog, and due to how I use the file system as a DB, there may or may not be files of meta data that exist.
spc476 2y ago
Turns out that mmap() is the memory allocator, nothing to do with files.
- throwway120385 2y ago
  I was going to point out that you can mmap() anonymous pages and use them directly as heap so this tracks.
- o11c 2y ago
  I did suspect that, but now the implication is that the memory allocator is not reusing memory even though it easily could. GLIBC used to have a fixed 128 KiB threshold, but since 2.25 (2017) it defaults to dynamically adjusting the threshold to prevent exactly this pattern of excess syscalls. A manual threshold disables this though! (This may be a microoptimization depending on how much real work you're doing between the syscalls, but there's also a good chance that it's easy to fix.)
yencabulator 2y ago
> * why are so many files opened but not closed? Guess: about 1/3 of those opens are unsuccessful.

by o11c 2y ago

spc476 2y ago

I do know how to answer most of the questions. The program is a gopher server. The list you see is the result of sending several hundred (sequential) requests to the server. The number of calls to open() vs. close() can be explained [1], so (to me) nothing unusual there. I don't directly call mmap(), munmap() or fstat(), so I'm assuming there's some code in some library I'm using that's doing those calls. And that's the assumption I would have with ioctl() as well (well, I now know better, but in investigating the issue originally I might just ignored ioctl() along with mmap()). There is a lot of reading, due to the requests (blog entries). Writing is to a socket. You are right in that there is too much epoll_ctl(), but it's unclear to me if I would have found the actual issue. I did do change the code to perform the real system call before registering, but that had little effect. [1] Not all requests are valid, for instance. In this case, it's a mirror of my blog, and due to how I use the file system as a DB, there may or may not be files of meta data that exist.

spc476 2y ago

Turns out that mmap() is the memory allocator, nothing to do with files.

throwway120385 2y ago
I was going to point out that you can mmap() anonymous pages and use them directly as heap so this tracks.
o11c 2y ago
I did suspect that, but now the implication is that the memory allocator is not reusing memory even though it easily could. GLIBC used to have a fixed 128 KiB threshold, but since 2.25 (2017) it defaults to dynamically adjusting the threshold to prevent exactly this pattern of excess syscalls. A manual threshold disables this though! (This may be a microoptimization depending on how much real work you're doing between the syscalls, but there's also a good chance that it's easy to fix.)

yencabulator 2y ago

> * why are so many files opened but not closed? Guess: about 1/3 of those opens are unsuccessful.