5 ms·
So, if a process is in an uninterruptible sleep, because say, it's doing i/o, say reading from disk, and furthermore, say it's using O_DIRECT, so the storage st
by smcameron 5y ago
So, if a process is in an uninterruptible sleep, because say, it's doing i/o, say reading from disk, and furthermore, say it's using O_DIRECT, so the storage stack is going to set things up to bypass the buffer cache and DMA directly into the process's memory, and then you rip away the process's memory and give it to another process... and then the DMA completes, and kaboom? The DMA just clobbered an unsuspecting process's memory?
That is, it was my understanding that the reason a process is in an uninterruptible sleep is generally because it's waiting for a DMA to complete, and if you were permitted to interrupt it, the DMA would eventually complete and clobber who knows what. Ripping the memory away from such a process would, to my first glance, seem to encounter the same problem -- how do you stop the pending DMA (which might already be in progress, but which might also take awhile to complete.) Whatever method, it would be device dependent, which makes it impractical (who's going to retrofit all the device drivers with DMA stopping APIs, and I'm sure there are many devices that have no way to stop pending DMAs, and anyway, maybe the DMA is already in progress, only half completed.) Maybe do it at the pci level, unmap DMA buffers. But many current drivers will generally assume that they never give their devices bad bus addresses, yet now the device is attempting DMA to a bad bus address (i.e. a suddenly unmapped bus address).
Well, I've been out of the linux driver game for awhile now, so perhaps I'm missing or forgetting something. Ripping memory out from under a process with pending DMA sounds pretty sketchy to me though.
But, if dumb old me can think of this, of course the kernel developers can also, and undoubtedly did. Wonder how it really works?
- phendrenad2 5y agoMayne usermode processes shouldn't have uninterruptible I/O access.
- scottlamb 5y ago> Mayne usermode processes shouldn't have uninterruptible I/O access. This isn't something userspace processes opt into. It's just how blocking read and write syscalls on filesystems work [1]. If you ever hit a bad block, you may notice that threads that touch it through the fs just hang and you can't recover or kill them. This is how it's always been on Linux and other Unix systems. I of course absolutely hate this and am 100% behind you if you're suggesting changing it. [1] With the exception of nfs if you set the "intr" mount option.
- neolog 5y agoThis also happens to mounted sshfs when the network goes down.
- phendrenad2 5y agoSo how do high-reliability Linux systems get around this? Find bad blocks before a process does? Or use non-blocking I/O? (How does Windows/VMS handle it, is another interesting question)
- monocasa 5y agoWindows NT is async at it's core, so it's handled more or less as Linux non-blocking I/O is, but across at least anything in the realm of the VFS rather being opt in by a FS implementation like Linux's non-blocking support.
- monocasa 5y agoI thought that the device will inc the reference count to that chunk of memory, so it won't be allocated to another process AFAIK.
- caf 5y agoThis is right. Those physical pages get pinned by the device. It's not like it's a new issue, anyway - physical backing pages can be dropped and re-used even if the process is still running, so the reference counting always had to exist.