4 ms·
Ask HN: How are Unix tools composable?
The second rule of UNIX is to make applications that can work with each other. All the stuff I read have to do with using text streams. My question is, even if it's a text stream, don't programs need to parse the text stream to get some semblance of a data structure they can work with? So if it can be any arbitrary text stream, how do they ensure composibility?
- PaulHoule 10y agoMost of them assume records separated with a record separator (typically \n) and fields separated with a field separator (often a comma, tabs, space.) That might not be a universal data model, but it is close enough for rock and roll. If you need more complex data structures than that, take a look at jq.
- dozzie 10y ago> So if it can be any arbitrary text stream, how do they ensure composibility? You got the composability thing wrong. The tools are composable because you feed them on STDIN or in command line arguments. You still need to do plenty of work to convert one format to another, and that's what awk, grep, sed, and other tools are for. The only two common (and just common, not universal) things are to split fields on whitespaces and to split records/entries on newline.
- informatimago 10y agoYes. Read http://web.mit.edu/~simsong/www/ugh.pdf http://web.mit.edu/~simsong/www/ugh.pdf Then read: http://www.catb.org/esr/writings/taoup/html/ http://www.catb.org/esr/writings/taoup/html/ You are responsible, as programmer or script writer, to ensure that the data you give to some tool is compatible! But notice also that the pipe is not the only IPC mechanism available on unix; there are a lot of other ways unix processes can communicate, from shared memory to external files, thru pipes, sockets, semaphores, signals and others. Each of these IPC mechanisms can be used to compose tools, as long as they are designed to use those mechanisms. In all cases, you need to document the specifications of the interfaces and data format.