7 ms·

Yes, file I/O can block. However, there is an assumption that file I/O will never block "indefinitely" -- unless something is severely broken, the kernel will a

Yes, file I/O can block. However, there is an assumption that file I/O will never block "indefinitely" -- unless something is severely broken, the kernel will always finish the operation in finite time probably measured in milliseconds at most. The same is not true of network communications, where you may be waiting for an event that never happens. There is a temptation to say that, well, milliseconds are a long time, so wouldn't we like to do this in a non-blocking way so we can work on other stuff in the meantime? But... consider this: Reads and writes of memory also may block. If you really think about it, the only real difference between main memory blocking and disk blocking is the amount of time they may block. And with modern SSDs that time difference is not as large as it used to be. So do you want to be able to access memory in a non-blocking way? Well... you can make the same logical arguments as you do with file I/O, but in practice, almost no one tries to do this. Instead, you separate work into threads, and let the CPU switch (hyper)threads whenever it needs to wait for memory. In fact, memory reads may very well block on disk, if you use swap! Given all this, it stops being so clear that async file I/O really makes sense. Meanwhile, as it happens, the Linux kernel was never really designed for async file I/O in the first place. When you perform file I/O, the kernel may need to execute filesystem driver code, and it does so within the same thread that invoked the operation from userspace. That filesystem code is blocking. For the kernel to deliver true async file I/O, either all this code needs to be rewritten to be non-blocking (which would probably slow it down in most cases!), or the kernel needs to start a thread behind the scenes to perform the work. But... you can just as easily start a thread in userspace. So... maybe just do that? (Or, the modern answer: Use io_uring, which is explicitly designed to allow a userspace thread to request work performed on a separate kernel thread, and get notified of completion later.)

loeg 4y ago
> For the kernel to deliver true async file I/O, either all this code needs to be rewritten to be non-blocking This is, I believe, the NT model.
deleted 4y ago
[deleted]
abiloe 4y ago
> If you really think about it, the only real difference between main memory blocking and disk blocking is the amount of time they may block. This is a somewhat confusing analysis you have here. Direct read/write from memory for all intents and purposes doesn't block. Why do you say that reads and writes may also block? The reason memory blocks is because it needs to page in or out from secondary storage - which makes this statement "the only real difference between main memory blocking and disk blocking is the amount of time they may block." not really true
- kentonv 4y ago
  > Direct read/write from memory for all intents and purposes doesn't block. Sure it does! Main memory is much slower than cache so on a cache miss the CPU has to stop and wait for main memory to respond. The CPU may even switch to executing some other thread in the meantime (that's what hyperthreading is). But if there isn't another hyperthread ready, the CPU sits idle, wasting resources. It's not a form of blocking implemented by the OS scheduler, but it's pretty similar conceptually. > The reason memory blocks is because it needs to page in or out from secondary storage Nope, that's not what I was referring to (other than in the line mentioning swap).
  - abiloe 4y ago
    > Sure it does! Main memory is much slower than cache so on a cache miss the CPU has to stop and wait for main memory to respond. The CPU may even switch to executing some other thread in the meantime (that's what hyperthreading is). Cache is a memory. And which cache, by the way? Even L1 cache on modern processors doesn't have 0 latency. And this is a rather poor way of describing hyperthreading - the CPU doesn't really "switch" - the context for the alternate process is already available and the resource stealing can occur for any kind of stall (including cache loads), not just memory. Calling this a "switch" suggesting it is like a context switch is very misleading. It's not similar conceptually. In any event, by this definition even a mispredicted branch or a divide becomes "blocking" - which sort of tortures any meaningful definition of blocking. The essential difference is - memory accesses to paged in memory (and branch mispredictions, cache misses) are not something you typically or reasonably trap outside of debugging. mmaps, swaps, disk I/O, network accesses are all something delegated to an OS - and at that point are orders of magnitude more expensive than even most NUMA memory accesses. I sort of see where you're coming from - but I don't think it's a useful point.
jeffbee 4y ago
io_uring just racked up another CVE, so I kinda feel that its severely under-designed nature will always haunt it. The idea that you can just hand off infinite amounts of work for the kernel to do on your behalf is pretty fundamentally broken. It is a concrete implementation of wishful thinking.
- vlovich123 4y ago
  That analysis would seem smart but let's try a game of Mad Libs: The Linux Kernel just racked up another CVE, so I kinda feel that its severely under-designed nature will always haunt it. KDE just racked up another CVE, so I kinda feel that its severely under-designed nature will always haunt it. Firefox just racked up another CVE, so I kinda feel that its severely under-designed nature will always haunt it. Chrome just racked up another CVE, so I kinda feel that its severely under-designed nature will always haunt it. Windows just racked up another CVE, so I kinda feel that its severely under-designed nature will always haunt it. Photoshop just racked up another CVE, so I kinda feel that its severely under-designed nature will always haunt it. All CPUs just rucked up another CVE, so I kinda feel that its severely under-designed nature will always haunt it. What's the theme? Racking up CVEs is something all software & hardware does. Mistakes can happen in design and in implementation and no one is immune. Using presence of CVEs as an indication of immaturity / fundamental design flaw isn't helpful. In fact, it's probably the opposite. Software that has no CVEs probably just means no one is paying attention to it. Sure, in a theoretical case maybe you've built a formal proof and translated that into a memory safe language somehow (& you assume you've made no mistakes modelling your entire system in your proof), then maybe. However, that encompasses 0% of all software. > The idea that you can just hand off infinite amounts of work for the kernel to do on your behalf is pretty fundamentally broken. It is a concrete implementation of wishful thinking How is that any different from a file descriptor? The kernel is free to setup limits on how much work you can have outstanding at any given time (now maybe those bits are missing right now, but it doesn't feel like an intractable problem).
- tankenmate 4y ago
  All "work" you want to do that interfaces with anything on an OS is handed off to the kernel; want to read a file? kernel, want to sleep for a while? kernel, etc. Besides things like network traffic is also asynchronous like io_uring (even if the socket() interfaces make it look somewhat synchronous). Outside of toy system asynchronicity is always a thing, especially when running on multiple cores. I kind of get where you are coming from but at the same time, the kernel always gets the last say, so as long as io_uring has a good design and implementation it will always be just as good or bad as the OS as a whole. Whether run of the mill programmers are up to the task of being able to properly conceptualise and use such an OS is probably not the same thing.
skissane 4y ago
> However, there is an assumption that file I/O will never block "indefinitely" -- unless something is severely broken, the kernel will always finish the operation in finite time probably measured in milliseconds at most. The same is not true of network communications, where you may be waiting for an event that never happens. Doesn’t this statement assume all filesystems are local? If the filesystem is remote (NFS, CIFS/SMB, WebDAV or SSH via FUSE, etc), then regular file IO is the same thing as network communication - and an NFS server going down (or losing connectivity to it) no more counts as “severely broken” than the same scenario for any other network protocol/application/service.
- izacus 4y ago
  That's not even true on devices that use slow flash storage, much less network mounted file systems. These assumptions tell a lot about why Linux so horribly falls apart when something like NFS lags a bit though.
- kentonv 4y ago
  A network filesystem is still expected to respond to reads and writes in a short finite time, assuming the network isn't broken. For an arbitrary network protocol, the amount of time you wait for an event is arbitrary and application-dependent. Even if everything is working exactly as intended, you could be waiting indefinitely for an event.
  - skissane 4y ago
    The network could be completely fine but the NFS server could have crashed (and maybe the backups were faulty so it is never coming back). That’s absolutely no different than if the same thing happened to an SSH server or a HTTP server or any other network service. The assumption is no more valid for NFS than it is for an arbitrary protocol.
throwaway09223 4y ago
> However, there is an assumption that file I/O will never block "indefinitely" No, there absolutely is not. > unless something is severely broken, the kernel will always finish the operation in finite time probably measured in milliseconds at most Systems get busy all the time and encounter delays larger than this all the time. I have fixed many supposedly low latency systems which incorrectly (as you suggest) assume that local disk writes ought to be fast because usually it hits a disk cache. It's very, very easy to break this assumption, like by copying a moderately sized file. You cannot assume that a disk write will finish quickly. That is not the guarantee. > Reads and writes of memory also may block. Not really, no. There are many orders of magnitude of difference. > If you really think about it, the only real difference between main memory blocking and disk blocking is the amount of time they may block. All syscalls take time, but the difference here is many orders of magnitude. If a syscall is expected to take a meaningful (or in the case of i/o, infinitely variable) amount of time we label it a blocking call. > In fact, memory reads may very well block on disk, if you use swap! System calls can't swap. High latency jobs will often mark themselves unswappable for exactly this reason, or systems will simply run swapless. > Given all this, it stops being so clear that async file I/O really makes sense. No. No no no. This is flatly wrong. You will ship broken systems if you do this and people will have to call someone who knows what they're doing to fix it. The best you can say is that statistically it won't happen often and maybe your users (or, management) won't notice until it's not your problem anymore. > But... you can just as easily start a thread in userspace. So... maybe just do that? Async i/o in glibc was historically implemented as simply a separate thread, fyi (prior to io_uring - I haven't looked in a decade). This is a form of async i/o.
- anonymoushn 4y ago
  > > Reads and writes of memory also may block. > Not really, no. There are many orders of magnitude of difference. If the kernel has locked your pages so it can merge or unmerge them under transparent huge pages or if you are using swap, page faults can take a really long time. People who care about async file I/O are probably already ensuring that these circumstances don't come up though.

loeg 4y ago

> For the kernel to deliver true async file I/O, either all this code needs to be rewritten to be non-blocking This is, I believe, the NT model.

deleted 4y ago

[deleted]

abiloe 4y ago

> If you really think about it, the only real difference between main memory blocking and disk blocking is the amount of time they may block. This is a somewhat confusing analysis you have here. Direct read/write from memory for all intents and purposes doesn't block. Why do you say that reads and writes may also block? The reason memory blocks is because it needs to page in or out from secondary storage - which makes this statement "the only real difference between main memory blocking and disk blocking is the amount of time they may block." not really true

kentonv 4y ago
> Direct read/write from memory for all intents and purposes doesn't block. Sure it does! Main memory is much slower than cache so on a cache miss the CPU has to stop and wait for main memory to respond. The CPU may even switch to executing some other thread in the meantime (that's what hyperthreading is). But if there isn't another hyperthread ready, the CPU sits idle, wasting resources. It's not a form of blocking implemented by the OS scheduler, but it's pretty similar conceptually. > The reason memory blocks is because it needs to page in or out from secondary storage Nope, that's not what I was referring to (other than in the line mentioning swap).
- abiloe 4y ago
  > Sure it does! Main memory is much slower than cache so on a cache miss the CPU has to stop and wait for main memory to respond. The CPU may even switch to executing some other thread in the meantime (that's what hyperthreading is). Cache is a memory. And which cache, by the way? Even L1 cache on modern processors doesn't have 0 latency. And this is a rather poor way of describing hyperthreading - the CPU doesn't really "switch" - the context for the alternate process is already available and the resource stealing can occur for any kind of stall (including cache loads), not just memory. Calling this a "switch" suggesting it is like a context switch is very misleading. It's not similar conceptually. In any event, by this definition even a mispredicted branch or a divide becomes "blocking" - which sort of tortures any meaningful definition of blocking. The essential difference is - memory accesses to paged in memory (and branch mispredictions, cache misses) are not something you typically or reasonably trap outside of debugging. mmaps, swaps, disk I/O, network accesses are all something delegated to an OS - and at that point are orders of magnitude more expensive than even most NUMA memory accesses. I sort of see where you're coming from - but I don't think it's a useful point.

jeffbee 4y ago

io_uring just racked up another CVE, so I kinda feel that its severely under-designed nature will always haunt it. The idea that you can just hand off infinite amounts of work for the kernel to do on your behalf is pretty fundamentally broken. It is a concrete implementation of wishful thinking.

vlovich123 4y ago
That analysis would seem smart but let's try a game of Mad Libs: The Linux Kernel just racked up another CVE, so I kinda feel that its severely under-designed nature will always haunt it. KDE just racked up another CVE, so I kinda feel that its severely under-designed nature will always haunt it. Firefox just racked up another CVE, so I kinda feel that its severely under-designed nature will always haunt it. Chrome just racked up another CVE, so I kinda feel that its severely under-designed nature will always haunt it. Windows just racked up another CVE, so I kinda feel that its severely under-designed nature will always haunt it. Photoshop just racked up another CVE, so I kinda feel that its severely under-designed nature will always haunt it. All CPUs just rucked up another CVE, so I kinda feel that its severely under-designed nature will always haunt it. What's the theme? Racking up CVEs is something all software & hardware does. Mistakes can happen in design and in implementation and no one is immune. Using presence of CVEs as an indication of immaturity / fundamental design flaw isn't helpful. In fact, it's probably the opposite. Software that has no CVEs probably just means no one is paying attention to it. Sure, in a theoretical case maybe you've built a formal proof and translated that into a memory safe language somehow (& you assume you've made no mistakes modelling your entire system in your proof), then maybe. However, that encompasses 0% of all software. > The idea that you can just hand off infinite amounts of work for the kernel to do on your behalf is pretty fundamentally broken. It is a concrete implementation of wishful thinking How is that any different from a file descriptor? The kernel is free to setup limits on how much work you can have outstanding at any given time (now maybe those bits are missing right now, but it doesn't feel like an intractable problem).
tankenmate 4y ago
All "work" you want to do that interfaces with anything on an OS is handed off to the kernel; want to read a file? kernel, want to sleep for a while? kernel, etc. Besides things like network traffic is also asynchronous like io_uring (even if the socket() interfaces make it look somewhat synchronous). Outside of toy system asynchronicity is always a thing, especially when running on multiple cores. I kind of get where you are coming from but at the same time, the kernel always gets the last say, so as long as io_uring has a good design and implementation it will always be just as good or bad as the OS as a whole. Whether run of the mill programmers are up to the task of being able to properly conceptualise and use such an OS is probably not the same thing.

skissane 4y ago

> However, there is an assumption that file I/O will never block "indefinitely" -- unless something is severely broken, the kernel will always finish the operation in finite time probably measured in milliseconds at most. The same is not true of network communications, where you may be waiting for an event that never happens. Doesn’t this statement assume all filesystems are local? If the filesystem is remote (NFS, CIFS/SMB, WebDAV or SSH via FUSE, etc), then regular file IO is the same thing as network communication - and an NFS server going down (or losing connectivity to it) no more counts as “severely broken” than the same scenario for any other network protocol/application/service.

izacus 4y ago
That's not even true on devices that use slow flash storage, much less network mounted file systems. These assumptions tell a lot about why Linux so horribly falls apart when something like NFS lags a bit though.
kentonv 4y ago
A network filesystem is still expected to respond to reads and writes in a short finite time, assuming the network isn't broken. For an arbitrary network protocol, the amount of time you wait for an event is arbitrary and application-dependent. Even if everything is working exactly as intended, you could be waiting indefinitely for an event.
- skissane 4y ago
  The network could be completely fine but the NFS server could have crashed (and maybe the backups were faulty so it is never coming back). That’s absolutely no different than if the same thing happened to an SSH server or a HTTP server or any other network service. The assumption is no more valid for NFS than it is for an arbitrary protocol.

throwaway09223 4y ago

> However, there is an assumption that file I/O will never block "indefinitely" No, there absolutely is not. > unless something is severely broken, the kernel will always finish the operation in finite time probably measured in milliseconds at most Systems get busy all the time and encounter delays larger than this all the time. I have fixed many supposedly low latency systems which incorrectly (as you suggest) assume that local disk writes ought to be fast because usually it hits a disk cache. It's very, very easy to break this assumption, like by copying a moderately sized file. You cannot assume that a disk write will finish quickly. That is not the guarantee. > Reads and writes of memory also may block. Not really, no. There are many orders of magnitude of difference. > If you really think about it, the only real difference between main memory blocking and disk blocking is the amount of time they may block. All syscalls take time, but the difference here is many orders of magnitude. If a syscall is expected to take a meaningful (or in the case of i/o, infinitely variable) amount of time we label it a blocking call. > In fact, memory reads may very well block on disk, if you use swap! System calls can't swap. High latency jobs will often mark themselves unswappable for exactly this reason, or systems will simply run swapless. > Given all this, it stops being so clear that async file I/O really makes sense. No. No no no. This is flatly wrong. You will ship broken systems if you do this and people will have to call someone who knows what they're doing to fix it. The best you can say is that statistically it won't happen often and maybe your users (or, management) won't notice until it's not your problem anymore. > But... you can just as easily start a thread in userspace. So... maybe just do that? Async i/o in glibc was historically implemented as simply a separate thread, fyi (prior to io_uring - I haven't looked in a decade). This is a form of async i/o.

anonymoushn 4y ago
> > Reads and writes of memory also may block. > Not really, no. There are many orders of magnitude of difference. If the kernel has locked your pages so it can merge or unmerge them under transparent huge pages or if you are using swap, page faults can take a really long time. People who care about async file I/O are probably already ensuring that these circumstances don't come up though.