3 ms·
There are also doubts internally about how much of this stuff will be actually useful to the rest of the world. This seems the wrong approach. If it's useful t
by pmjordan 17y ago
There are also doubts internally about how much of this stuff will be actually useful to the rest of the world.
This seems the wrong approach. If it's useful to google, it might not be useful to the rest of the world in exactly that form, but I'm sure outside kernel devs would have opinions on it and could suggest a version that helps everyone.
Google makes a lot of use of the out-of-memory (OOM) killer to pare back overloaded systems. That can create trouble, though, when processes holding mutexes encounter the OOM killer. Mike wonders why the kernel tries so hard, rather than just failing allocation requests when memory gets too tight.
This is something that really bothers me about Linux too, and I'm not google, I run a workstation and some servers. Processes with runaway memory allocations can easily grind the system to a halt by causing crazy amounts of swapping. If you don't catch it in time, you basically need to hard reset. This shouldn't be possible from user space.
Mike concluded with a couple of "interesting problems." One of those is that Google would like a way to pin filesystem metadata in memory. The problem here is being able to bound the time required to service I/O requests. The time required to read a block from disk is known, but if the relevant metadata is not in memory, more than one disk I/O operation may be required. That slows things down in undesirable ways. Google is currently getting around this by reading file data directly from raw disk devices in user space,
Ouch!
but they would like to stop doing that.
No kidding.
- psranga 17y agoFor the memory allocation problem, do you have any ideas about how other OS's address this problem? How about ulimit? Could that be useful?