6 ms·

This may be from 2012, but it's surprising just how much of the specific advice still applies. Grep, sed, awk, xargs etc are still the canonical list of shell

by yaseer 5y ago

This may be from 2012, but it's surprising just how much of the specific advice still applies. Grep, sed, awk, xargs etc are still the canonical list of shell tools for debugging, and have been for decades. Makes you realise, learning Unix tools will likely be a better long term time investment than the latest trendy framework. I wish I could go back in time and tell that to my younger self...

gjkood 5y ago
Couldn't agree more! When I am forced to use a Windows environment I find myself reaching out to these evergreen tools in many situations and is frustrated by having to find workarounds to these tools that should be ubiquitous tools in any OS. Everything I learned almost 30 years ago still holds true today when it comes to the basics. And yes I know of Cygwin, MKS toolkit etc, but trying to convince corporate IT in a Microsoft Centric world to install these usually hits a brick wall.
- ______-_-______ 5y ago
  Thankfully git is everywhere these days, and git on windows comes with most of the coreutils. I hope IT isn't telling you you're not allowed to use source control. Microsoft owns GitHub after all
  - capableweb 5y ago
    Microsoft Azure also offers Blockchain solutions, that doesn't mean every company using Microsoft's C# is generally pro-blockchain, just as it doesn't mean every company is using Git because they are a Microsoft-shop.
- odonnellryan 5y ago
  Sometimes nothing beats grep, but using a Windows environment I usually just go to Python to create a quick script to handle this stuff. Lots of times it is really nice to have all your "debugging scripts" in a repo anyway. Once you have that it isn't hard to set up a basic Python environment and have your scripts there.
  - 0des 5y ago
    When your logs are big enough python doesn't really compare in terms of speed but maybe in terms of speed to write the thing you need to query them if you don't know how to use the aforementioned tools
- mciancia 5y ago
  Eh, forcing people to use Windows in big corps is such PITA :(
- pjmlp 5y ago
  Or learn the ways to achieve the same with PowerShell, and Technet utilities. Not every OS needs to be an UNIX clone.
charlieflowers 5y ago
What are some outstanding resources to learn from?
- PeterWhittaker 5y ago
  Honestly, I think the best way to learn these tools is to use them. After starting at BNR in 1990, I sat for a couple of days with Sobell's Practical UNIX on my lap, doing every example and every exercise; since them, I've stayed at the command line as much as possible. I've looked periodically for other, more recent resources, at least in part to answer questions such as yours, but I've never found anything that packed the punch of that early Sobell, not even later Sobell's. If you want to direct the learning, though, I'd recommend figuring out how to do something with grep, e.g., then do the same with sed, awk, cut, etc. For example: grep myuserid /etc/passwd sed -n '/myuserid/p' /etc/passwd awk '/myuserid/ { print } etc. Then make it more interesting: repeat those with 'grep -I MYUSERID /etc/passwd'. Then make it even more interesting: print the 4th field from each matching line. (Hint: man cut.) Come up with at least two or three different ways of doing this with regex's, then make those regex's as compact as possible. Then make it a lot more interesting: print the two lines before and the three lines after each pattern. Once you do this for a while, you will get used to thinking in terms of filters and regex and things will start to come more naturally for you. Also consider doing all of those exercises from within vim, use :g... commands (or any other alternative you can think of).
- daemoncoder 5y ago
  https://www.oreilly.com/library/view/unix-power-tools/0596003307/ https://www.oreilly.com/library/view/unix-power-tools/059600... I used this a great deal, when I first started out on serious UNIX tasks (1996+)
- dsanchez97 5y ago
  I liked the Missing Semester by MIT. It provides a first principle style for learning some of the foundational tools. https://www.youtube.com/playlist?list=PLyzOVJj3bHQuloKGG59rS43e29ro7I57J https://www.youtube.com/playlist?list=PLyzOVJj3bHQuloKGG59rS....
bobbylarrybobby 5y ago
This is a fun read on the topic: https://adamdrake.com/command-line-tools-can-be-235x-faster-than-your-hadoop-cluster.html https://adamdrake.com/command-line-tools-can-be-235x-faster-...

by yaseer 5y ago

gjkood 5y ago

Couldn't agree more! When I am forced to use a Windows environment I find myself reaching out to these evergreen tools in many situations and is frustrated by having to find workarounds to these tools that should be ubiquitous tools in any OS. Everything I learned almost 30 years ago still holds true today when it comes to the basics. And yes I know of Cygwin, MKS toolkit etc, but trying to convince corporate IT in a Microsoft Centric world to install these usually hits a brick wall.

______-_-______ 5y ago
Thankfully git is everywhere these days, and git on windows comes with most of the coreutils. I hope IT isn't telling you you're not allowed to use source control. Microsoft owns GitHub after all
- capableweb 5y ago
  Microsoft Azure also offers Blockchain solutions, that doesn't mean every company using Microsoft's C# is generally pro-blockchain, just as it doesn't mean every company is using Git because they are a Microsoft-shop.
odonnellryan 5y ago
Sometimes nothing beats grep, but using a Windows environment I usually just go to Python to create a quick script to handle this stuff. Lots of times it is really nice to have all your "debugging scripts" in a repo anyway. Once you have that it isn't hard to set up a basic Python environment and have your scripts there.
- 0des 5y ago
  When your logs are big enough python doesn't really compare in terms of speed but maybe in terms of speed to write the thing you need to query them if you don't know how to use the aforementioned tools
mciancia 5y ago
Eh, forcing people to use Windows in big corps is such PITA :(
pjmlp 5y ago
Or learn the ways to achieve the same with PowerShell, and Technet utilities. Not every OS needs to be an UNIX clone.

charlieflowers 5y ago

What are some outstanding resources to learn from?

PeterWhittaker 5y ago
Honestly, I think the best way to learn these tools is to use them. After starting at BNR in 1990, I sat for a couple of days with Sobell's Practical UNIX on my lap, doing every example and every exercise; since them, I've stayed at the command line as much as possible. I've looked periodically for other, more recent resources, at least in part to answer questions such as yours, but I've never found anything that packed the punch of that early Sobell, not even later Sobell's. If you want to direct the learning, though, I'd recommend figuring out how to do something with grep, e.g., then do the same with sed, awk, cut, etc. For example: grep myuserid /etc/passwd sed -n '/myuserid/p' /etc/passwd awk '/myuserid/ { print } etc. Then make it more interesting: repeat those with 'grep -I MYUSERID /etc/passwd'. Then make it even more interesting: print the 4th field from each matching line. (Hint: man cut.) Come up with at least two or three different ways of doing this with regex's, then make those regex's as compact as possible. Then make it a lot more interesting: print the two lines before and the three lines after each pattern. Once you do this for a while, you will get used to thinking in terms of filters and regex and things will start to come more naturally for you. Also consider doing all of those exercises from within vim, use :g... commands (or any other alternative you can think of).
daemoncoder 5y ago
https://www.oreilly.com/library/view/unix-power-tools/0596003307/ https://www.oreilly.com/library/view/unix-power-tools/059600... I used this a great deal, when I first started out on serious UNIX tasks (1996+)
dsanchez97 5y ago
I liked the Missing Semester by MIT. It provides a first principle style for learning some of the foundational tools. https://www.youtube.com/playlist?list=PLyzOVJj3bHQuloKGG59rS43e29ro7I57J https://www.youtube.com/playlist?list=PLyzOVJj3bHQuloKGG59rS....

bobbylarrybobby 5y ago

This is a fun read on the topic: https://adamdrake.com/command-line-tools-can-be-235x-faster-than-your-hadoop-cluster.html https://adamdrake.com/command-line-tools-can-be-235x-faster-...