16 ms·

Better Bash History (2012)

267 points by Pete_D 7y ago

tapoxi 7y ago
highly recommended for fellow ctrl-r addicts: https://github.com/junegunn/fzf https://github.com/junegunn/fzf
- umvi 7y ago
  These ones are pretty good too: https://github.com/cantino/mcfly https://github.com/cantino/mcfly https://github.com/dvorka/hstr https://github.com/dvorka/hstr
- graton 7y ago
  I also recommend FZF. Really very handy for doing CTRL-R searches. I also use it for doing interactive git rebases. I have an alias for doing a fuzzy rebase interactive (frbi): frbi = !git rebase -i $(git log --pretty=oneline --color=always | fzf --ansi | cut -d ' ' -f1)^
  - kbd 7y ago
    This is a great idea, thanks for it! I've done similar things with using fzf to git add, git diff, and so on, but I hadn't thought of using it to pick a commit to rebase from. I made a few changes so I figured I'd post them back here in case you're interested: pick-commit = # equivalent to your git log... cut... rif = "!f() { rev=$(git pick-commit); [[ $rev ]] && git ri $rev~; }; f" "rif" = "rebase interactive fzf/fuzzyfind". The function and test just makes it exit cleanly if you escape out of the filter instead of giving "fatal: invalid upstream '~'" (and I changed it from ^ to ~ because I think ~ is usually what one should use over ^, but I'm willing to be corrected). And obviously I already have 'ri' aliased to 'rebase -i'. While I'm here, here are my similar git aliases for adding, diffing, etc. files using fzf: pick-status-files = "!f() { git status -z | xargs -0n1 | cut -c4- | fzi --print0 | xargs -0to git "$@"; }; f" af = !git pick-status-files add # and so on for diff, difftool, checkout, etc. Oh, and 'fzi' (fuzzy-inline) is a small shell script that just calls: fzf --height 30% --reverse --multi "$@"
    - graton 7y ago
      Thanks for that. I like your improvements to my idea too! :)
      - kbd 7y ago
        Yay. Btw, one more improvement. TIL xargs will skip blank inputs (depending on bsd/gnu xargs, flags etc...), so my version above: rif = "!f() { rev=$(git pick-commit); [[ $rev ]] && git ri $rev~; }; f" can just be: rif = !git pick-commit | xargs -tI% git ri %~
    - girishso 7y ago
      That looks great. Do you have your dotfiles shared by any chance?
      - kbd 7y ago
        Sure: https://github.com/kbd/setup https://github.com/kbd/setup Here's my git config with a bajillion aliases: https://github.com/kbd/setup/blob/master/HOME/.config/git/config https://github.com/kbd/setup/blob/master/HOME/.config/git/co... Since git commands are one of the most common things to type, I alias all short git aliases like so: https://github.com/kbd/setup/blob/fd1f826a9365895b7a3b8b5c585e5154863d87cd/HOME/bin/shell_sources/aliases.sh#L126 https://github.com/kbd/setup/blob/fd1f826a9365895b7a3b8b5c58... (link to particular commit so line numbers stay correct). That way my workflow looks like: $ ga (fuzzy find files to add) $ gcm "commit message" etc.
    - kbd 7y ago
      Can't edit this anymore. pick-status-files = "!f() { git status -z | xargs -0n1 | cut -c4- | fzi --print0 | xargs -0to git "$@"; }; f" can just be: pick-status-files = !git status -z | xargs -0n1 | cut -c4- | fzi --print0 | xargs -0t git I'd recently converted it from a standalone script so I just converted it verbatim into a function.
- barbuk 7y ago
  fzf is the most fun and productive tool added in my dotfiles in recent years. Mix it with https://github.com/rupa/z https://github.com/rupa/z or https://github.com/skywind3000/z.lua https://github.com/skywind3000/z.lua for fuzzysearch/cd And you can do cool scripts, like: - search your vim edited files with highlighted preview: https://github.com/BarbUk/dotfiles/blob/master/shell/completion#L136 https://github.com/BarbUk/dotfiles/blob/master/shell/complet... - search and open your chrome history: https://github.com/BarbUk/dotfiles/blob/master/shell/completion#L155 https://github.com/BarbUk/dotfiles/blob/master/shell/complet...
aaronharnly 7y ago
I set up my bash config to store history separately for each working directory (ie separate history files that I store under a $HOME/.bash_history_d directory) – so as I cd into a project directory, the history is populated with the commands that I've run within that directory. I find this super-helpful to remind me of the context for projects – cd in, then up-arrow to whatever that command is I usually run to launch this server, grep this logfile, etc. There are some downsides – sometimes I want to run the same command in directory X and also in subdirectory X/Y, but I love doing it this way. Anyone else done something similar?
- zwieback 7y ago
  That's clever, I'd like that. Can you have another key to recall the global history, e.g. all the combined histories like you would normally see?
- dllthomas 7y ago
  I do something similar, but instead of current directory I have a notion of context that is carried by an environment variable set in a GNU screen process (and therefore inherited by its children) which drives logic in my .bashrc. https://github.com/dlthomas/config-files https://github.com/dlthomas/config-files
  - dual_basis 7y ago
    Fish does this by default (and much more as well).
    - dllthomas 7y ago
      Could you elaborate? I can read docs, but it's not clear to me what correspondences you have in mind. Specifically, I'm not sure what "this" is that fish does, as I described interaction of multiple pieces of software and I don't think fish includes a terminal multiplexer?
      - jackbravo 7y ago
mistrial9 7y ago
can anyone please explain how BASH history on current Ubuntu, is different when logging in from different desktop machines, to the same login ? the dot-local directory or dot-gconf directory is part of this ? consternation..
O_H_E 7y ago
>By default, Bash only records a session to the `.bash_history` file on disk when the session terminates. This means that if you crash or your session terminates improperly, you lose the history up to that point. I kept loosing my history consistently on my new installation because of this. I usually put my laptop to sleep, and pretty much never intentionally terminate my x-session except on kernel updates. But because its a cheapo laptop with a crappy battery, I would occasionally run out of battery when asleep or it just wont wake up.
- barrkel 7y ago
  I have a fairly expensive ThinkPad and it also runs out if battery if asleep for more than about 4 days, and frequently freezes on suspend or resume when it's been both suspended and resumed docked and undocked. (Linux, of course.)
thenewwazoo 7y ago
Because this comes up every so often, I'll post this link again: I wrote a little widget that stores your bash history in a sqlite db. I haven't touched it in a while because I now rely on fzf and this doesn't integrate with it, though maybe I could make it do... https://github.com/thenewwazoo/bash-history-sqlite https://github.com/thenewwazoo/bash-history-sqlite
- chubot 7y ago
  This is cool, somebody recently suggested that I add history to sqlite in Oil [1]. I looked at your code but I didn't see how it integrates with bash? Most people seem to be using the PROMPT_COMMAND hook for stuff like this -- are you using something else, or did I miss it? https://github.com/oilshell/oil/issues/320 https://github.com/oilshell/oil/issues/320 I will add PROMPT_COMMAND to Oil, but I'm curious if there is something else I should add to support end-user customization. One thing that might be interesting is to support a hook for the ! syntax, so the sqlite DB could be searched. I think that would solve some of the problems you mention in the code's comments, and maybe the fzf issue. (Although that literally just occurred to me, I don't promise to do this :) ) I haven't looked at how fzf works exactly. But if you want to chat about it please chime in on Github :) [1] Latest status: Success With the Interactive Shell http://www.oilshell.org/blog/2019/02/05.html http://www.oilshell.org/blog/2019/02/05.html . Oil runs a bunch of interactive programs like bash-completion, virtualenv, and git-prompt!
  - skissane 7y ago
    > This is cool, somebody recently suggested that I add history to sqlite in Oil I think a shell with an embedded sqlite, which makes sqlite accessible to scripts as shell builtins, and in which command line history is stored in a sqlite database instead of a plain text file, would be pretty cool. You could store command history (and maybe even other stuff) in something like $HOME/.shell_db As well as just a list of lines, other useful information could be stored for each line, such as current directory, timestamp, username (maybe I su to root and am running commands as root but still going into my own account's history database), hostname (if $HOME is on NFS, putting aside the issue that databases, including sqlite, don't work well over NFS due to file locking issues), etc.
kbd 7y ago
Bash's broken history is what finally got me to switch to Zsh. The history behavior you want is: 1. each line of history is appended incrementally, but 2. each shell session only has access to its own history unless you specifically reload the history into your session. Every combination of history commands in bash didn't do the right thing[1], but Zsh's works as desired. [1] eg. putting 'history -a' in PROMPT_COMMAND as the article recommends appends your session's entire history each time, IIRC.
- cortesoft 7y ago
  See, what I want from my history is every command I have ever run, so that ctrl r can find anything I have done.
  - kbd 7y ago
    Yes what I wrote doesn't exclude that. The only distinction is that you generally don't want all your sessions' histories interleaved. I.e. when you up-arrow, you only want your current session's history, without random things from other active sessions in there too. If you do want "everything you've ever done" including things from other sessions that started after your current session, it's easy to reload your history into your session. I have a 'rlh' (reload history) alias for that.
    - jackewiehose 7y ago
      You always only have your current session history in each bash (unless you reload the history manually) so there is no interleaving.
      - kbd 7y ago
        Yeah I know, I was just making a distinction from cortesoft's "I want everything I've ever done" in my history. I was saying "so do I, except I want active sessions separate" (which as you point out, they are by default).
- jackewiehose 7y ago
stilist 7y ago
Since everybody has their own way of handling it, mine is * Generate a new history file for every session (https://github.com/stilist/dotfiles/blob/master/dot_sh/session/history.sh https://github.com/stilist/dotfiles/blob/master/dot_sh/sessi...) * A 'histgrep' script that searches across all the files and highlights matches (https://github.com/stilist/dotfiles/blob/master/dot_sh/bin/executable_histgrep https://github.com/stilist/dotfiles/blob/master/dot_sh/bin/e...)
MarcScott 7y ago
I've had this in my .bashrc for years. I thought it came from this post judging from my commments, but looks different, so I wonder whether it came from HN comments when this was previously posted. ## Better bash history # Avoid duplicates export HISTCONTROL=ignoredups:erasedups # When the shell exits, append to the history file instead of overwriting it shopt -s histappend # After each command, append to the history file and reread it export PROMPT_COMMAND="${PROMPT_COMMAND:+$PROMPT_COMMAND$'\n'}history -a; history -c; history -r"
mrzool 7y ago
I love this blog and learned so much about the Unix toolkit and the philosophy behind it by reading it. Too bad it doesn’t seem to be much active anymore, but there’s a lot of great material in the archives that pretty much never gets old since its focused mostly on standard Unix software. The author (Tom Ryder) is an very talented writer with a clear and concise style. A couple of plugs for your consideration: - I integrated all of these history options in my project Sensible Bash [1], an attempt at saner Bash defaults - I made an ebook [2] out of a series which appeared on this blog, titled “Unix as IDE”. [1]: https://github.com/mrzool/bash-sensible https://github.com/mrzool/bash-sensible [2]: https://github.com/mrzool/unix-as-ide https://github.com/mrzool/unix-as-ide
deleted 7y ago
[deleted]
pkaye 7y ago
We have some servers where the user home directory is on nfs. To keep separate history for each server, I add the following to .profile. export HISTFILE="${HOME}/.bash_history.$(hostname)"
bhaak 7y ago
I just recently stumbled over https://spin.atomicobject.com/2016/05/28/log-bash-history/ https://spin.atomicobject.com/2016/05/28/log-bash-history/ Using PROMPT_COMMAND to log every command in a separate timestamped log file. This allows you to use a much better format than bash history has to be in. Easily readable to see what you've been doing and extremly efficient to grep for. Has been discussed on HN before: https://news.ycombinator.com/item?id=11806553 https://news.ycombinator.com/item?id=11806553
jefftk 7y ago
I do: promptFunc() { # right before prompting for the # next command, save the previous # command in a file. echo "$(date +%Y-%m-%d--%H-%M-%S) \ $(hostname) $PWD $(history 1)" \ >> ~/.full_history } PROMPT_COMMAND=promptFunc This is in addition to standard bash history, and is much more reliable. Being able to see not only what command I ran but what directory I was in and when I ran it is very useful. (I wrote about this here: https://www.jefftk.com/p/you-should-be-logging-shell-history https://www.jefftk.com/p/you-should-be-logging-shell-history)
- deeg 7y ago
  I do something very similar (just saved in files by month) and it has saved my bacon a few times. I was recently asked to re-run a process that I ran years ago. It would have taken me hours to re-discover how to run the process but instead took a few minutes after grepping the commands. Every developer should do this.
- 314 7y ago
  Yeah, this is how I do it - although I append the hostname to the end of the filename and sync the collection between my machines so that I can grep for history on any machine. Edit: went to check and apparently I've overcomplicated this somewhat (it's about ten years old) but it is nice if empty lines are not logged. It's an old habit to ctrl-c then hit return repeatedly if a terminal is not responding. PROMPT_COMMAND=storehist storehist () { CMDCNT=`history 1 | sed -e 's/\w* *$.*$/\1/'`; if [ "$CMDCNT" != "$LASTCMDIND" ] && [ -n "$LASTCMDIND" ]; then DATE=`date '+%Y%b%d %H%M'`; if [ "$LASTCMDIND" != "$CMDCNT" ]; then echo "$DATE $HOSTNAME:$BASHTTY $CMDCNT" >> ~/.custom_history; fi; LASTCMDIND="$CMDCNT"; fi; LASTCMDIND="$CMDCNT" }
naggie 7y ago
I have a global bash history, having set up bash/zsh to [immediately share without any annoying race conditions.][2] I use FZF mapped to CTRL+R to search through my history. The unique thing is that I use a [python script (in my dotfiles)][1] to clean the history in a more advanced way than is possible with HISTIGNORE. The script removes duplicates, leaving the most recent copy and removes lines matching several regexes. [1]: https://github.com/naggie/dotfiles/blob/master/scripts/cleanup-history https://github.com/naggie/dotfiles/blob/master/scripts/clean... [2]: https://github.com/naggie/dotfiles/blob/master/home/.bashrc#L26 https://github.com/naggie/dotfiles/blob/master/home/.bashrc#...
- n8henrie 7y ago
  I was recently planning to write something just like this. Thanks for sharing.
- IronBacon 7y ago
  If one is only interested to remove duplicates (your script does more) I've found this one liner in a StackExchange post: awk '!x[$0]++' ~/.zsh_history
  - pwagland 7y ago
    Note that this only keeps the first copy, not the most recent copy. To make it so that it only keeps the most recent, you would need to run this through `tac` twice.
commandersaki 7y ago
Probably the most useful thing I've done regarding bash history: https://eli.thegreenplace.net/2016/persistent-history-in-bash-redux/ https://eli.thegreenplace.net/2016/persistent-history-in-bas...
ConSeannery 7y ago
To echo what everyone else is saying, get FZF. Most of these linux productivity things are just some dumb pet project that solves a specific problem that a specific person finds annoying. FZF is not this - it makes something you do hundreds of times a day (history and file opening on zsh, bash, fish, whatever) indisputably better. If you find yourself sitting there tapping ctrl+r multiple times like an idiot for some thing that may or may not be there, do your stubborn beardy ass a favour and install fzf. You can implement some or all of the tips in this HN post, it'll make fzf results better.
- shriek 7y ago
  Yup, I tried the context aware history for couple of days and it got annoying pretty fast. I have everything in one file which I can just FZF reverse search and bam. No need for infinite up arrows.
- kbumsik 7y ago
  Yeah, these kinds of hacks usually end up with unmaintainable mess when cumulated. Just use fzf.
- mceachen 7y ago
  https://github.com/junegunn/fzf https://github.com/junegunn/fzf for the uninitiated
ausjke 7y ago
that's exactly what my setting is for history, nice post. The only difference is that I unset HISTCONTROL, don't recall why I did that. shopt -s histappend is the default already I believe. shopt -s cmdhist is also the default now. HISTSIZE=-1 will be unlimited, which is fine for today's computers.
zonidjan 7y ago
PROMPT_COMMAND is probably already used for something else by your distro. HISTFILESIZE and HISTSIZE can be set to -1 for unlimited. Read man bash.
- bhaak 7y ago
  HISTFILESIZE and HISTSIZE don't solve the abysmal bash history file handling when you have more than one bash open. I would also like to know which distro clobbers user settable env variables.
omeid2 7y ago
I use fzf for bash history. Fuzzy history search and much better TUI. https://github.com/junegunn/fzf https://github.com/junegunn/fzf
tenken 7y ago
Meh. I prefer to leverage SQL. I have yet to find a better way to store and search bash history than: https://github.com/tkf/rash https://github.com/tkf/rash More recent bash sqlite history attempts all seem to fall short than this implementation, they store less metrics or have other caveats.
romka2 7y ago
I remember reading this article back when I was still using Bash. I've finally got the history I like only after switching to ZSH. You can get decent history in ZSH just by setting the right options. If you want great history, you'll need to write a bit of code. Right now my history works as follows. History is written to disk after every command. Up and Down keys go through the local history (from the same session). Ctrl+Up and Ctrl+Down go through the shared history (from all sessions). Ctrl+R also uses shared history (it's easy to add another binding for local history but I don't have it). Pressing Up/Ctrl+Up after typing something will go over history entries that have the matching prefix. For example, `git<Up>` will show the last command from the current session that starts with `git`. Here's my config: https://old.reddit.com/r/zsh/comments/bsa224/how_to_setup_a_better_history/eon23cl https://old.reddit.com/r/zsh/comments/bsa224/how_to_setup_a_.... In addition, my history is stored in Git. History from the local machine comes first, and from other machines second. This is fairly easy to do it Bash, too.

267 points by Pete_D 7y ago

tapoxi 7y ago

highly recommended for fellow ctrl-r addicts: https://github.com/junegunn/fzf https://github.com/junegunn/fzf

umvi 7y ago
These ones are pretty good too: https://github.com/cantino/mcfly https://github.com/cantino/mcfly https://github.com/dvorka/hstr https://github.com/dvorka/hstr
graton 7y ago
I also recommend FZF. Really very handy for doing CTRL-R searches. I also use it for doing interactive git rebases. I have an alias for doing a fuzzy rebase interactive (frbi): frbi = !git rebase -i $(git log --pretty=oneline --color=always | fzf --ansi | cut -d ' ' -f1)^
- kbd 7y ago
  This is a great idea, thanks for it! I've done similar things with using fzf to git add, git diff, and so on, but I hadn't thought of using it to pick a commit to rebase from. I made a few changes so I figured I'd post them back here in case you're interested: pick-commit = # equivalent to your git log... cut... rif = "!f() { rev=$(git pick-commit); [[ $rev ]] && git ri $rev~; }; f" "rif" = "rebase interactive fzf/fuzzyfind". The function and test just makes it exit cleanly if you escape out of the filter instead of giving "fatal: invalid upstream '~'" (and I changed it from ^ to ~ because I think ~ is usually what one should use over ^, but I'm willing to be corrected). And obviously I already have 'ri' aliased to 'rebase -i'. While I'm here, here are my similar git aliases for adding, diffing, etc. files using fzf: pick-status-files = "!f() { git status -z | xargs -0n1 | cut -c4- | fzi --print0 | xargs -0to git "$@"; }; f" af = !git pick-status-files add # and so on for diff, difftool, checkout, etc. Oh, and 'fzi' (fuzzy-inline) is a small shell script that just calls: fzf --height 30% --reverse --multi "$@"
  - graton 7y ago
    Thanks for that. I like your improvements to my idea too! :)
    - kbd 7y ago
      Yay. Btw, one more improvement. TIL xargs will skip blank inputs (depending on bsd/gnu xargs, flags etc...), so my version above: rif = "!f() { rev=$(git pick-commit); [[ $rev ]] && git ri $rev~; }; f" can just be: rif = !git pick-commit | xargs -tI% git ri %~
  - girishso 7y ago
    That looks great. Do you have your dotfiles shared by any chance?
    - kbd 7y ago
      Sure: https://github.com/kbd/setup https://github.com/kbd/setup Here's my git config with a bajillion aliases: https://github.com/kbd/setup/blob/master/HOME/.config/git/config https://github.com/kbd/setup/blob/master/HOME/.config/git/co... Since git commands are one of the most common things to type, I alias all short git aliases like so: https://github.com/kbd/setup/blob/fd1f826a9365895b7a3b8b5c585e5154863d87cd/HOME/bin/shell_sources/aliases.sh#L126 https://github.com/kbd/setup/blob/fd1f826a9365895b7a3b8b5c58... (link to particular commit so line numbers stay correct). That way my workflow looks like: $ ga (fuzzy find files to add) $ gcm "commit message" etc.
  - kbd 7y ago
    Can't edit this anymore. pick-status-files = "!f() { git status -z | xargs -0n1 | cut -c4- | fzi --print0 | xargs -0to git "$@"; }; f" can just be: pick-status-files = !git status -z | xargs -0n1 | cut -c4- | fzi --print0 | xargs -0t git I'd recently converted it from a standalone script so I just converted it verbatim into a function.
barbuk 7y ago
fzf is the most fun and productive tool added in my dotfiles in recent years. Mix it with https://github.com/rupa/z https://github.com/rupa/z or https://github.com/skywind3000/z.lua https://github.com/skywind3000/z.lua for fuzzysearch/cd And you can do cool scripts, like: - search your vim edited files with highlighted preview: https://github.com/BarbUk/dotfiles/blob/master/shell/completion#L136 https://github.com/BarbUk/dotfiles/blob/master/shell/complet... - search and open your chrome history: https://github.com/BarbUk/dotfiles/blob/master/shell/completion#L155 https://github.com/BarbUk/dotfiles/blob/master/shell/complet...

aaronharnly 7y ago

I set up my bash config to store history separately for each working directory (ie separate history files that I store under a $HOME/.bash_history_d directory) – so as I cd into a project directory, the history is populated with the commands that I've run within that directory. I find this super-helpful to remind me of the context for projects – cd in, then up-arrow to whatever that command is I usually run to launch this server, grep this logfile, etc. There are some downsides – sometimes I want to run the same command in directory X and also in subdirectory X/Y, but I love doing it this way. Anyone else done something similar?

zwieback 7y ago
That's clever, I'd like that. Can you have another key to recall the global history, e.g. all the combined histories like you would normally see?
dllthomas 7y ago
I do something similar, but instead of current directory I have a notion of context that is carried by an environment variable set in a GNU screen process (and therefore inherited by its children) which drives logic in my .bashrc. https://github.com/dlthomas/config-files https://github.com/dlthomas/config-files
- dual_basis 7y ago
  Fish does this by default (and much more as well).
  - dllthomas 7y ago
    Could you elaborate? I can read docs, but it's not clear to me what correspondences you have in mind. Specifically, I'm not sure what "this" is that fish does, as I described interaction of multiple pieces of software and I don't think fish includes a terminal multiplexer?
    - jackbravo 7y ago

mistrial9 7y ago

can anyone please explain how BASH history on current Ubuntu, is different when logging in from different desktop machines, to the same login ? the dot-local directory or dot-gconf directory is part of this ? consternation..

O_H_E 7y ago

>By default, Bash only records a session to the `.bash_history` file on disk when the session terminates. This means that if you crash or your session terminates improperly, you lose the history up to that point. I kept loosing my history consistently on my new installation because of this. I usually put my laptop to sleep, and pretty much never intentionally terminate my x-session except on kernel updates. But because its a cheapo laptop with a crappy battery, I would occasionally run out of battery when asleep or it just wont wake up.

barrkel 7y ago
I have a fairly expensive ThinkPad and it also runs out if battery if asleep for more than about 4 days, and frequently freezes on suspend or resume when it's been both suspended and resumed docked and undocked. (Linux, of course.)

thenewwazoo 7y ago

Because this comes up every so often, I'll post this link again: I wrote a little widget that stores your bash history in a sqlite db. I haven't touched it in a while because I now rely on fzf and this doesn't integrate with it, though maybe I could make it do... https://github.com/thenewwazoo/bash-history-sqlite https://github.com/thenewwazoo/bash-history-sqlite

chubot 7y ago
This is cool, somebody recently suggested that I add history to sqlite in Oil [1]. I looked at your code but I didn't see how it integrates with bash? Most people seem to be using the PROMPT_COMMAND hook for stuff like this -- are you using something else, or did I miss it? https://github.com/oilshell/oil/issues/320 https://github.com/oilshell/oil/issues/320 I will add PROMPT_COMMAND to Oil, but I'm curious if there is something else I should add to support end-user customization. One thing that might be interesting is to support a hook for the ! syntax, so the sqlite DB could be searched. I think that would solve some of the problems you mention in the code's comments, and maybe the fzf issue. (Although that literally just occurred to me, I don't promise to do this :) ) I haven't looked at how fzf works exactly. But if you want to chat about it please chime in on Github :) [1] Latest status: Success With the Interactive Shell http://www.oilshell.org/blog/2019/02/05.html http://www.oilshell.org/blog/2019/02/05.html . Oil runs a bunch of interactive programs like bash-completion, virtualenv, and git-prompt!
- skissane 7y ago
  > This is cool, somebody recently suggested that I add history to sqlite in Oil I think a shell with an embedded sqlite, which makes sqlite accessible to scripts as shell builtins, and in which command line history is stored in a sqlite database instead of a plain text file, would be pretty cool. You could store command history (and maybe even other stuff) in something like $HOME/.shell_db As well as just a list of lines, other useful information could be stored for each line, such as current directory, timestamp, username (maybe I su to root and am running commands as root but still going into my own account's history database), hostname (if $HOME is on NFS, putting aside the issue that databases, including sqlite, don't work well over NFS due to file locking issues), etc.

kbd 7y ago

Bash's broken history is what finally got me to switch to Zsh. The history behavior you want is: 1. each line of history is appended incrementally, but 2. each shell session only has access to its own history unless you specifically reload the history into your session. Every combination of history commands in bash didn't do the right thing[1], but Zsh's works as desired. [1] eg. putting 'history -a' in PROMPT_COMMAND as the article recommends appends your session's entire history each time, IIRC.

cortesoft 7y ago
See, what I want from my history is every command I have ever run, so that ctrl r can find anything I have done.
- kbd 7y ago
  Yes what I wrote doesn't exclude that. The only distinction is that you generally don't want all your sessions' histories interleaved. I.e. when you up-arrow, you only want your current session's history, without random things from other active sessions in there too. If you do want "everything you've ever done" including things from other sessions that started after your current session, it's easy to reload your history into your session. I have a 'rlh' (reload history) alias for that.
  - jackewiehose 7y ago
    You always only have your current session history in each bash (unless you reload the history manually) so there is no interleaving.
    - kbd 7y ago
      Yeah I know, I was just making a distinction from cortesoft's "I want everything I've ever done" in my history. I was saying "so do I, except I want active sessions separate" (which as you point out, they are by default).
jackewiehose 7y ago

stilist 7y ago

Since everybody has their own way of handling it, mine is * Generate a new history file for every session (https://github.com/stilist/dotfiles/blob/master/dot_sh/session/history.sh https://github.com/stilist/dotfiles/blob/master/dot_sh/sessi...) * A 'histgrep' script that searches across all the files and highlights matches (https://github.com/stilist/dotfiles/blob/master/dot_sh/bin/executable_histgrep https://github.com/stilist/dotfiles/blob/master/dot_sh/bin/e...)

MarcScott 7y ago

I've had this in my .bashrc for years. I thought it came from this post judging from my commments, but looks different, so I wonder whether it came from HN comments when this was previously posted. ## Better bash history # Avoid duplicates export HISTCONTROL=ignoredups:erasedups # When the shell exits, append to the history file instead of overwriting it shopt -s histappend # After each command, append to the history file and reread it export PROMPT_COMMAND="${PROMPT_COMMAND:+$PROMPT_COMMAND$'\n'}history -a; history -c; history -r"

mrzool 7y ago

I love this blog and learned so much about the Unix toolkit and the philosophy behind it by reading it. Too bad it doesn’t seem to be much active anymore, but there’s a lot of great material in the archives that pretty much never gets old since its focused mostly on standard Unix software. The author (Tom Ryder) is an very talented writer with a clear and concise style. A couple of plugs for your consideration: - I integrated all of these history options in my project Sensible Bash [1], an attempt at saner Bash defaults - I made an ebook [2] out of a series which appeared on this blog, titled “Unix as IDE”. [1]: https://github.com/mrzool/bash-sensible https://github.com/mrzool/bash-sensible [2]: https://github.com/mrzool/unix-as-ide https://github.com/mrzool/unix-as-ide

deleted 7y ago

[deleted]

pkaye 7y ago

We have some servers where the user home directory is on nfs. To keep separate history for each server, I add the following to .profile. export HISTFILE="${HOME}/.bash_history.$(hostname)"

bhaak 7y ago

I just recently stumbled over https://spin.atomicobject.com/2016/05/28/log-bash-history/ https://spin.atomicobject.com/2016/05/28/log-bash-history/ Using PROMPT_COMMAND to log every command in a separate timestamped log file. This allows you to use a much better format than bash history has to be in. Easily readable to see what you've been doing and extremly efficient to grep for. Has been discussed on HN before: https://news.ycombinator.com/item?id=11806553 https://news.ycombinator.com/item?id=11806553

jefftk 7y ago

I do: promptFunc() { # right before prompting for the # next command, save the previous # command in a file. echo "$(date +%Y-%m-%d--%H-%M-%S) \ $(hostname) $PWD $(history 1)" \ >> ~/.full_history } PROMPT_COMMAND=promptFunc This is in addition to standard bash history, and is much more reliable. Being able to see not only what command I ran but what directory I was in and when I ran it is very useful. (I wrote about this here: https://www.jefftk.com/p/you-should-be-logging-shell-history https://www.jefftk.com/p/you-should-be-logging-shell-history)

deeg 7y ago
I do something very similar (just saved in files by month) and it has saved my bacon a few times. I was recently asked to re-run a process that I ran years ago. It would have taken me hours to re-discover how to run the process but instead took a few minutes after grepping the commands. Every developer should do this.
314 7y ago
Yeah, this is how I do it - although I append the hostname to the end of the filename and sync the collection between my machines so that I can grep for history on any machine. Edit: went to check and apparently I've overcomplicated this somewhat (it's about ten years old) but it is nice if empty lines are not logged. It's an old habit to ctrl-c then hit return repeatedly if a terminal is not responding. PROMPT_COMMAND=storehist storehist () { CMDCNT=`history 1 | sed -e 's/\w* *$.*$/\1/'`; if [ "$CMDCNT" != "$LASTCMDIND" ] && [ -n "$LASTCMDIND" ]; then DATE=`date '+%Y%b%d %H%M'`; if [ "$LASTCMDIND" != "$CMDCNT" ]; then echo "$DATE $HOSTNAME:$BASHTTY $CMDCNT" >> ~/.custom_history; fi; LASTCMDIND="$CMDCNT"; fi; LASTCMDIND="$CMDCNT" }

naggie 7y ago

I have a global bash history, having set up bash/zsh to [immediately share without any annoying race conditions.][2] I use FZF mapped to CTRL+R to search through my history. The unique thing is that I use a [python script (in my dotfiles)][1] to clean the history in a more advanced way than is possible with HISTIGNORE. The script removes duplicates, leaving the most recent copy and removes lines matching several regexes. [1]: https://github.com/naggie/dotfiles/blob/master/scripts/cleanup-history https://github.com/naggie/dotfiles/blob/master/scripts/clean... [2]: https://github.com/naggie/dotfiles/blob/master/home/.bashrc#L26 https://github.com/naggie/dotfiles/blob/master/home/.bashrc#...

n8henrie 7y ago
I was recently planning to write something just like this. Thanks for sharing.
IronBacon 7y ago
If one is only interested to remove duplicates (your script does more) I've found this one liner in a StackExchange post: awk '!x[$0]++' ~/.zsh_history
- pwagland 7y ago
  Note that this only keeps the first copy, not the most recent copy. To make it so that it only keeps the most recent, you would need to run this through `tac` twice.

commandersaki 7y ago

Probably the most useful thing I've done regarding bash history: https://eli.thegreenplace.net/2016/persistent-history-in-bash-redux/ https://eli.thegreenplace.net/2016/persistent-history-in-bas...

ConSeannery 7y ago

To echo what everyone else is saying, get FZF. Most of these linux productivity things are just some dumb pet project that solves a specific problem that a specific person finds annoying. FZF is not this - it makes something you do hundreds of times a day (history and file opening on zsh, bash, fish, whatever) indisputably better. If you find yourself sitting there tapping ctrl+r multiple times like an idiot for some thing that may or may not be there, do your stubborn beardy ass a favour and install fzf. You can implement some or all of the tips in this HN post, it'll make fzf results better.

shriek 7y ago
Yup, I tried the context aware history for couple of days and it got annoying pretty fast. I have everything in one file which I can just FZF reverse search and bam. No need for infinite up arrows.
kbumsik 7y ago
Yeah, these kinds of hacks usually end up with unmaintainable mess when cumulated. Just use fzf.
mceachen 7y ago
https://github.com/junegunn/fzf https://github.com/junegunn/fzf for the uninitiated

ausjke 7y ago

that's exactly what my setting is for history, nice post. The only difference is that I unset HISTCONTROL, don't recall why I did that. shopt -s histappend is the default already I believe. shopt -s cmdhist is also the default now. HISTSIZE=-1 will be unlimited, which is fine for today's computers.

zonidjan 7y ago

PROMPT_COMMAND is probably already used for something else by your distro. HISTFILESIZE and HISTSIZE can be set to -1 for unlimited. Read man bash.

bhaak 7y ago
HISTFILESIZE and HISTSIZE don't solve the abysmal bash history file handling when you have more than one bash open. I would also like to know which distro clobbers user settable env variables.

omeid2 7y ago

I use fzf for bash history. Fuzzy history search and much better TUI. https://github.com/junegunn/fzf https://github.com/junegunn/fzf

tenken 7y ago

Meh. I prefer to leverage SQL. I have yet to find a better way to store and search bash history than: https://github.com/tkf/rash https://github.com/tkf/rash More recent bash sqlite history attempts all seem to fall short than this implementation, they store less metrics or have other caveats.

romka2 7y ago

I remember reading this article back when I was still using Bash. I've finally got the history I like only after switching to ZSH. You can get decent history in ZSH just by setting the right options. If you want great history, you'll need to write a bit of code. Right now my history works as follows. History is written to disk after every command. Up and Down keys go through the local history (from the same session). Ctrl+Up and Ctrl+Down go through the shared history (from all sessions). Ctrl+R also uses shared history (it's easy to add another binding for local history but I don't have it). Pressing Up/Ctrl+Up after typing something will go over history entries that have the matching prefix. For example, `git<Up>` will show the last command from the current session that starts with `git`. Here's my config: https://old.reddit.com/r/zsh/comments/bsa224/how_to_setup_a_better_history/eon23cl https://old.reddit.com/r/zsh/comments/bsa224/how_to_setup_a_.... In addition, my history is stored in Git. History from the local machine comes first, and from other machines second. This is fairly easy to do it Bash, too.