4 ms·
Yes, I don't understand how people end up with assertions that the filename is a require argument. At least we've got /dev/stdin or /proc/self/fd/0 as workaroun
by jeffbee 6y ago
Yes, I don't understand how people end up with assertions that the filename is a require argument. At least we've got /dev/stdin or /proc/self/fd/0 as workarounds.
- benibela 6y agofiles might be faster, because you can mmap them?
- f311a 6y agoMost people work on compressed JSON lines files. Sometimes they are stored on s3. Files do not give flexibility. When using jq, I can do a lot of things: aws s3 cp s3://bucket/file.json.gz - | zcat | head | jq .field | sort
- jerf 6y agoAnother common thing you can do is accept a generic stream as input, but have some code that penetrates the abstraction a bit to see what kind of stream it is, and if it is a file or something, do something special with it to go even faster. This way, you start with something maximally useful up front, and easy to use, but you can optimize things based on details as you go. That's how Go's static file web server works. It serves streams, but if you happen to io.Copy to that stream with something that is also an ∗os.File on Linux, it can use the sendfile call in the kernel instead. (A downside of making it so transparent is that if you wrap that stream with something you may not realize that you've wrecked the optimization because it no longer unwraps to an ∗os.File but whatever your wrapper is, but, well, nothing's perfect.)
- Spivak 6y agoWhich is sorta-kinda the same thing as jq .field <(aws s3 cp s3://bucket/file.json.gz - | zcat | head) | sort which is more annoying to type but works.
- dullgiulio 6y agoRedirection is not a parameter, meaning that would still not work with this Q tool.
- jraph 6y agoThis is not a simple redirection. cmd <(subcmd) is a bashism that redirects the output of command subcmd to something that looks like a regular file to command cmd. Command cmd receive a path at the place the <(subcmd) syntax is used. Different from cmd < f, which redirects the contents of file f to cmd's input. So, this should work :-)
- caymanjim 6y agoMuch Unix software today is written by people who really don't appreciate or understand Unix. It leads to things like Homebrew (early on, at least) completely taking over and breaking /usr/local; to command-line utilities using a single dash (-) precending both short and long option names (granted, --long-opts is a GNUism, but it's a well-established standard); commands that output color by default even when the output isn't a tty; etc. It's not hard to fix things like this, but it exemplifies a lack of familiarity with the Unix command line. There are an enormous number of tools out there that only exist because people don't know how to chain together basic 1970s Unix text-processing tools in a pipeline.
- jeffbee 6y agoThe `go` program uses -longopts and I think it would be hard to argue that Rob Pike lacks an appreciation of Unix traditions.
- bfrydl 6y agoI would argue that Go's design as a whole is characterized by an attitude of ignoring established ideas for no other reason than that they think they know better.
- thatjamesdude 6y agoSomething being established is not a grand argument for it's usage. The reasons it got established are relavent, and if you feel the end result of said establishment is obtuse or inane, why would you use it? That's not to say Go's decisions to toss some established practices are "wise" or "sagely", just that broad acceptance is not a criteria they seemed concerned with. Which is fine. >they think they know better. It's safe to say Rob Pike is not clueless or without experience in unix tooling. You should listen to some of his experiences and thoughts with designing Go [0]. I don't always agree with him, [but it's very baseless to suggest he makes decisions on the grounds that they were his, not they have merrit.] Edit to clarify: [He makes decisions on merrit over authority] [0]: https://changelog.com/gotime/100 https://changelog.com/gotime/100