5 ms·
Introduction to Named Pipes
- eesmith 6y agoThis makes me wonder if named pipes is a solution to a problem I'm having. I use a 3rd party took with a Python API which only takes filenames (not stdin and not Python file objects). It natively understands gzip and non-gzip files. I would like it to support zstandard compression. I think I can set up a named pipe with ["zstdcat", "filename.zstd"] as input, directed to a named pipe, and have the 3rd party toolkit read from that named pipe, as a regular text file. But I'm a bit confused about the coordination. 1) how do I find a valid file system location? I can use tempfile to get a writable file location, but that might be on a network scratch disk. Is that going to be okay, or do I need some mechanism to be able to specify a local writable filesystem? 2) I can start the zstdcat via subprocess, but how do I know when to wait() on it, given that the 3rd party doesn't (easily) tell me when it's done. (I guess I can put a wrapper around the API, and check with the reader finishes.) 3) How do I manage removing the named pipe? Is there a way to do that automatically when both sides close? Or is there some other mechanism I could do? I can do all this via wrappers of various sorts, but the coordination seems complex, and I'm not sure what will happen if I ^C in the middle of things. Has anyone here done something like this before? I don't see any relevant package on PyPI for this sort of wrapping.