3 ms·
Basic message: big parallel I/O systems need to shift responsibility for maintaining file state, moving away from the operating system and towards the applicati
by benlorenzetti 9y ago
Basic message: big parallel I/O systems need to shift responsibility for maintaining file state, moving away from the operating system and towards the application.
POSIX API has pretty simple view of file state with open(), read(), write(), and close(), but this interface does not scale well concurrency-wise.
- notacoward 9y agoThat part scales fine. There are other problems with the POSIX open/read/write model, such as being too strict wrt consistency and too loose wrt ordering and durability, but it scales OK. It's the metadata/namespace operations that are hard to scale. I've been working on distributed filesystems for a long time. If all I had to deal with was reads and writes I'd be a happy (OK, less angry) man.
- benlorenzetti 9y agoI trust your experience over my own in this area; just reading this piece with curiosity. That makes sense, metadata and namespace operations would be difficult when distributed. But isn't the graph in the article specifically about file handle state maintained by the OS? I guess the graph is weird though it wants to be logarithmic but the power of the series keeps decreasing...what type of graph is that?