6 ms·

* 'comm -3' is a quick way to do a set difference from the command line. * Newer versions of sort have an option for running sorts in parallel. If you're using

* 'comm -3' is a quick way to do a set difference from the command line. * Newer versions of sort have an option for running sorts in parallel. If you're using an older sort, you can split the files, sort them individually with GNU parallel, and use --merge to combine them. * If you have scripts that read data files, process them, and output more files, consider using a Makefile. * tmux is a good way to leave a development session on a server that you come back to at the start of the day, or that you run long-running processes in. * Setting a soft ulimit system-wide is a good way to avoid accidentally running the machine out of memory, causing your C libraries to be paged out to disk, and usually requiring a reboot. Since it's soft, you can override it at any time. * strace and gdb can be run on just about anything to see what it's doing. If it's an interpreted language, you can often attach to a live process, call a function to invoke some interpreted code, and not have to kill & restart the process to fix a bug or oversight. Or you can attach to a Postgres worker process to get a stack trace, which often tells you why your query is slow if it runs so long you can't get an explain analyze. * Look in /proc to find tons of useful information about running processes and settings. For example, you can tell how many filehandles a process has open, and sometimes what file it corresponds to. Just yesterday, a grub update caused a server to boot the kernel with the command line replaced with 2 random characters and a newline, which would be harder to debug without /proc/cmdline. * pushd and popd are useful in shell scripts to temporarily change directories without forgetting where you were.

falsedan 9y ago
> If you have scripts that read data files, process them, and output more files, consider using a Makefile. Dear sweet hypnotoad, don't use Makefiles for this. Scripts in a pipeline are perfectly well suited for ETL. They have the advantage of using the same language as the command language (Makefile is not shell, and when it differs it's a significant surprise). Plus, you can drop a script into a dir in your PATH and it will work in any project. Use make if you have a project which expensively computes assets and reusing partial outputs is possible. > pushd and popd are useful in shell scripts to temporarily change directories without forgetting where you were. I would start with cd - # go back to where I just came from Which handles most of the pushd/popd use cases (outside of writing a script). Really, there's a whole separate set of skills for effective interactive shell use & writing great bash scripts (e.g. parameter expansion and history manipulation).
- deleted 9y ago
  [deleted]
- kingosticks 9y ago
  I'd stick with pushd and popd every time, it's considerably more expressive.
  - falsedan 9y ago
    > considerably more expressive is that an advantage? Do you have the time to explain this a bit more? I feel that new users need less expressiveness, to avoid decision overload, and keeping one automatic directory save point is easier to mentally manage than a stack of them. I do recommend using pushd/popd in shell scripts (always) and interactively (if you must), but I think 'cd -' should be the first thing you introduce to newcomers w.r.t tracking working directory changes.
    - jerf 9y ago
      Speaking for myself, I'm often using pushd & popd precisely because I expect to potentially be moving around in the new directory too, and cd - will just take me to the previous dir. pushd & popd don't result in me having even a bit of stress about typing another cd command. Your mileage will vary. That said, I don't disagree that cd - is a good introduction to the idea that there's more to the cd command than meets the eye. (I'm also a big fan of the CDPATH variable, despite its issues.)
    - godd2 9y ago
      'cd -' only saves one previous directory, which is held in $OLDPWD. pushd can store an arbitrary history in its stack. And of course, $OLDPWD will change if you hop around after a cd, but the pushd/popd stack will persist.
      - falsedan 9y ago
        That's true, but I rather would have liked to hear how the expressiveness of pushd/popd was superior to cd - (especially with respect to a newcomer internalizing all the weird coreutils & bash things).

falsedan 9y ago

> If you have scripts that read data files, process them, and output more files, consider using a Makefile. Dear sweet hypnotoad, don't use Makefiles for this. Scripts in a pipeline are perfectly well suited for ETL. They have the advantage of using the same language as the command language (Makefile is not shell, and when it differs it's a significant surprise). Plus, you can drop a script into a dir in your PATH and it will work in any project. Use make if you have a project which expensively computes assets and reusing partial outputs is possible. > pushd and popd are useful in shell scripts to temporarily change directories without forgetting where you were. I would start with cd - # go back to where I just came from Which handles most of the pushd/popd use cases (outside of writing a script). Really, there's a whole separate set of skills for effective interactive shell use & writing great bash scripts (e.g. parameter expansion and history manipulation).

deleted 9y ago
[deleted]
kingosticks 9y ago
I'd stick with pushd and popd every time, it's considerably more expressive.
- falsedan 9y ago
  > considerably more expressive is that an advantage? Do you have the time to explain this a bit more? I feel that new users need less expressiveness, to avoid decision overload, and keeping one automatic directory save point is easier to mentally manage than a stack of them. I do recommend using pushd/popd in shell scripts (always) and interactively (if you must), but I think 'cd -' should be the first thing you introduce to newcomers w.r.t tracking working directory changes.
  - jerf 9y ago
    Speaking for myself, I'm often using pushd & popd precisely because I expect to potentially be moving around in the new directory too, and cd - will just take me to the previous dir. pushd & popd don't result in me having even a bit of stress about typing another cd command. Your mileage will vary. That said, I don't disagree that cd - is a good introduction to the idea that there's more to the cd command than meets the eye. (I'm also a big fan of the CDPATH variable, despite its issues.)
  - godd2 9y ago
    'cd -' only saves one previous directory, which is held in $OLDPWD. pushd can store an arbitrary history in its stack. And of course, $OLDPWD will change if you hop around after a cd, but the pushd/popd stack will persist.
    - falsedan 9y ago
      That's true, but I rather would have liked to hear how the expressiveness of pushd/popd was superior to cd - (especially with respect to a newcomer internalizing all the weird coreutils & bash things).