4 ms·

You just taught me something cool as someone who loves commandline I’m definitely going to try this and save this as a gist for future reference

lucb1e 6y ago
What was it that you didn't know yet? As an aside, a friend of mine has already arranged (as a joke) the rights to compile and remix any bash snippets I post in chat groups in a blog or book. Maybe I should write more about this, but then, it's probably most helpful exactly because I just drop it in places rather than writing a comprehensive work that nobody sits down to read. Still, I'm curious what the syntax/feature/pattern is that you found useful :)
- TheSoftwareGuy 6y ago
  Personally, I didn't know about the ( ... commands ... ) > some_file.txt syntax. Like, I knew about file redirection, but I didn't know you could group commands like that and redirect all their output with one operator. I would have had to do a bunch of appending with `>>`
- mraza007 6y ago
  Hey if you create a book or blog I would love to check out your bash scripts You should definitely try mdbook if you want to keep it in a book format as it offers good fulltext search And I found the entire script interesting I would definitely try this once I get to my laptop as I’m currently surfing HN on my phone
- mraza007 6y ago
  If you got more bash hacks or one liners or cool commands for automation I would love to check it out !!!
  - lucb1e 6y ago
    Three tools that I think are less widely known compared to how useful they are: jq, progress, and pv. `pv` (pipe view) is rather simple but I use it for viewing a lot and sometimes rate-limiting transfers. Usage is simple: `curl huge-file | pv | some-operation` will show you the progress (okay, curl already does... but pv works for any pipe input, and pv can work in line mode with -l so you can see how many lines (records, presumably) it processed). `progress` is even simpler! Just type progress in a random terminal while, in another, you are running an operation like cp, gzip, etc. $ progress [10123] bzip2 /tmp/test 32.5% (127.0 MiB / 390.6 MiB) Use -m to monitor instead of doing a one-shot. This will also allow it to calculate the remaining time (for anything running at more than a few kilobytes per second). I also love progress just because it's such a hack. It hooks into the process and monitors file operations. Simple but effective. Somehow it works super well because it manages to ignore everything irrelevant and only shows you the file handle you care about (you can give it hints if necessary, I've used that maybe once in ten years). `jq` takes JSON data and either pretty-prints (default) or operates on it: $ echo '{"whoami": "root"}' | jq { "whoami": "root" } # imagine colored output above $ echo '{"whoami": "root"}' | jq .whoami "root" $ echo '{"users": [{"name": "abc", ...}, ...]}' | jq '.users[].name' # prints the name of all users in this array $ curl localhost/api/v1/users | jq '.users[] | "\(.name) \(.age)"' | sort -n -k 2 | tail -1 && echo "is our oldest user!"
    - autorun 6y ago
      I also recommend `pypyp` python package. You can pipe to python like, $ seq 1 10 | pyp "sum(map(int, lines))" 55 or along with `jq` and `pup` (html parser) $ curl https://public-api.wordpress.com/rest/v1.1/sites/username.wordpress.com/posts | jq -r ".posts.content" | pyp "html.unescape(x)" | pup a attr{href}
    - mraza007 6y ago
      Hey I just get enthusiastic about bash and commandline in general as these tools have saved my hours of work compared to using gui And I love your enthusiasm in the comment and things you just taught me today. I’m definitely going to take a note of this and add to my TIL Notes so I can always refer back to this I just wanted to thank you for teaching me something totally new today

lucb1e 6y ago

What was it that you didn't know yet? As an aside, a friend of mine has already arranged (as a joke) the rights to compile and remix any bash snippets I post in chat groups in a blog or book. Maybe I should write more about this, but then, it's probably most helpful exactly because I just drop it in places rather than writing a comprehensive work that nobody sits down to read. Still, I'm curious what the syntax/feature/pattern is that you found useful :)

TheSoftwareGuy 6y ago
Personally, I didn't know about the ( ... commands ... ) > some_file.txt syntax. Like, I knew about file redirection, but I didn't know you could group commands like that and redirect all their output with one operator. I would have had to do a bunch of appending with `>>`
mraza007 6y ago
Hey if you create a book or blog I would love to check out your bash scripts You should definitely try mdbook if you want to keep it in a book format as it offers good fulltext search And I found the entire script interesting I would definitely try this once I get to my laptop as I’m currently surfing HN on my phone
mraza007 6y ago
If you got more bash hacks or one liners or cool commands for automation I would love to check it out !!!
- lucb1e 6y ago
  Three tools that I think are less widely known compared to how useful they are: jq, progress, and pv. `pv` (pipe view) is rather simple but I use it for viewing a lot and sometimes rate-limiting transfers. Usage is simple: `curl huge-file | pv | some-operation` will show you the progress (okay, curl already does... but pv works for any pipe input, and pv can work in line mode with -l so you can see how many lines (records, presumably) it processed). `progress` is even simpler! Just type progress in a random terminal while, in another, you are running an operation like cp, gzip, etc. $ progress [10123] bzip2 /tmp/test 32.5% (127.0 MiB / 390.6 MiB) Use -m to monitor instead of doing a one-shot. This will also allow it to calculate the remaining time (for anything running at more than a few kilobytes per second). I also love progress just because it's such a hack. It hooks into the process and monitors file operations. Simple but effective. Somehow it works super well because it manages to ignore everything irrelevant and only shows you the file handle you care about (you can give it hints if necessary, I've used that maybe once in ten years). `jq` takes JSON data and either pretty-prints (default) or operates on it: $ echo '{"whoami": "root"}' | jq { "whoami": "root" } # imagine colored output above $ echo '{"whoami": "root"}' | jq .whoami "root" $ echo '{"users": [{"name": "abc", ...}, ...]}' | jq '.users[].name' # prints the name of all users in this array $ curl localhost/api/v1/users | jq '.users[] | "\(.name) \(.age)"' | sort -n -k 2 | tail -1 && echo "is our oldest user!"
  - autorun 6y ago
    I also recommend `pypyp` python package. You can pipe to python like, $ seq 1 10 | pyp "sum(map(int, lines))" 55 or along with `jq` and `pup` (html parser) $ curl https://public-api.wordpress.com/rest/v1.1/sites/username.wordpress.com/posts | jq -r ".posts.content" | pyp "html.unescape(x)" | pup a attr{href}
  - mraza007 6y ago
    Hey I just get enthusiastic about bash and commandline in general as these tools have saved my hours of work compared to using gui And I love your enthusiasm in the comment and things you just taught me today. I’m definitely going to take a note of this and add to my TIL Notes so I can always refer back to this I just wanted to thank you for teaching me something totally new today