5 ms·
1) `pgrep` is a standard utility that does what his `psgrep` does and much much more. 2) htop is a cpu and memory hog -- every time I've used it I noticed it t
by niggler 14y ago
1) `pgrep` is a standard utility that does what his `psgrep` does and much much more.
2) htop is a cpu and memory hog -- every time I've used it I noticed it takes 6+% CPU time
3) there's an awk trick to do the `sort | uniq` recommendation that works on 10+GB files (single pass):
awk '!x[$0]++'
4) Passwordless keys are dangerous -- use ssh-agent to save the password of the keys
- vvpan 14y agoOn my computer htop consumes about the same amount of resources as top, just tested.
- carlesfe 14y agoNot trying to contradict you, just some explanations 1. I prefer 'psgrep' because it covers 99% of my use cases for pgrep (ps axuf | grep $NAME) 2. htop is very nice, come on! Would not let it running on the background for hours, but it's nicer than top 3. Note taken, thanks! 4. Is ssh-agent really safer than using passwordless keys? Just asking, I'm curious
- niggler 14y ago1. `man pgrep` is your friend (you save two greps, and TBH the `grep -v grep` should be a hint that there's a better way) 2. In my experience on Debian (granted this was in 2010), there is a noticeable performance difference between `htop` and `top`. 4. ssh-agent stores the password in memory and is erased on reboot. OTOH If you use a passwordless key file, anyone can use it if they have the key.
- jfb 14y ago1. I always went with % ps auwx | grep '[f]oo' | awk '{ print $2; }'
- jerf 14y agoUsing ssh-agent also means you can practically put a very large pass phrase on your SSH key, because you'll only type it infrequently. Good luck brute forcing my passphrase.
- npsimons 14y agoI'm pretty sure ssh-agent doesn't store the password, but the private key. Also, the fact that it supports timed expire (and can be setup to drop keys upon events such as screen lock) make it a wiser choice than passwordless keys.
- gingerlime 14y agoThat's correct. And ssh-agent doesn't give access to the private key either, only to perform operations like signing. The only way to extract the key is to search it in the process memory, which I believe would requires root level access.
- ominous_prime 14y agore: 4: not only is an ssh agent by far safer, but most agents now allow you to set a timeout on a key, so it's not indefinitely saved in memory. A passwordless key gives anyone with acces to that file, access to the login associated with it. If that file is inadvertently exposed (oops, checked it into github...), any machine you have a login on must be considered compromised.
- edwintorok 14y agossh-agent and 'ssh -A' is also useful if you have to login to one machine to access another, without having to copy your private key to the first machine. For example if you login remotely to a machine, and want to access a git repository on another: $ eval `ssh-agent -s` $ ssh-add ~/.ssh/id_<yourkey> $ ssh -A <firstserver> you@firstserver$ git clone git+ssh://<secondserver>/path/to/repository
- jackalope 14y agoI use agent forwarding often, but you still need to be careful, especially if you forward your agent to a machine not under your control. From the ssh man page: Agent forwarding should be enabled with caution. Users with the ability to bypass file permissions on the remote host (for the agent's UNIX-domain socket) can access the local agent through the forwarded connection. An attacker cannot obtain key material from the agent, however they can perform operations on the keys that enable them to authenticate using the identities loaded into the agent. Consider using a dedicated key for each of those circumstances, set sane defaults in your ~/.ssh/config on all machines, and be very careful about what ends up in any of your ~/.ssh/known_hosts files, as they provide a road map to other destinations.
- pepve 14y agoThe ssh_config HashKnownHosts option hashes the contents of the known_hosts file, making it intractable to get a list of hosts. But of course your shell history will still provide it.
- JoeAcchino 14y ago3) if all the lines of the 10+GB file are actually unique, wouldn't awk keep the whole file in RAM? For files larger than my RAM could this leave my system unresponsive because it's thrashing on swap?
- niggler 14y agoThe sort | uniq method literally needs to sort the file and pipe it to uniq, a far more memory-intensive operation than the single-pass awk check. You can write your own hash function in AWK if you think you may overstep memory, but of course you risk hash collisions. It's a tradeoff. I tried it on a 1U server with 24GB ram a few years ago and found that the sort was thrashing at the 10GB file size while AWK handled it easily
- carlesfe 14y agoYes, awk does some real black bagic. It's awesome how it can parse really, really big files.
- stevvooe 14y agoSort will actually externally sort blocks into temp files and merge them. Adjusting this block size can help with thrashing. Awk may still be better for uniquification.
- enigmo 14y agoWhat about 'sort -u' ?
- stevvooe 14y agoI'm not sure. I haven't closely studied the difference between each algorithm. My guess would be that sort -u would perform better as the data set gets larger with a good block size setting because it does do an external sort. Cardinality would also affect the performance. If the unique set handily fits in memory, an external sort on a large data set wouldn't be very efficient.