4 ms·

I often have multiple concurrent sessions of terminal open, which leads to messed up history interactions. One solution I've encountered (but have not tested ye

I often have multiple concurrent sessions of terminal open, which leads to messed up history interactions. One solution I've encountered (but have not tested yet) is zsh-histdb [0]. It stores sessions' history to a SQLite db instead of a single appended file, with extra metadata about when commands were run, what session, etc. If you've already got your .zshrc file open to mess with the history settings it might be worth checking out that tool while you're at it. 0: https://github.com/larkery/zsh-histdb https://github.com/larkery/zsh-histdb

outcoldman 4y ago
If somebody is interested in macos app for storing your history (bash,zsh,fish) and syncing it with icloud, I have built an app ShellHistory, v2 is available via TestFlight if you want to try it. https://loshadki.app/blog/2022-09-22-public-beta-shellhistory-v2/ https://loshadki.app/blog/2022-09-22-public-beta-shellhistor... (monterey and ventura) And this is a link to v1 https://loshadki.app/shellhistory/ https://loshadki.app/shellhistory/ (big sur+)
- InvaderFizz 4y ago
  Been using this for about a year, well worth the $12. Thanks man! Saves me finding a command I ran but I don't remember when, or in which of my 50+ open and not committed shells. Also the exit code of every command is super helpful.
  - outcoldman 4y ago
    Thank you for feedback! Appreciate to see a user from HN :D
  - UI_at_80x24 4y ago
    >...or in which of my 50+ open and not committed shells... I suffered this frustration too, but adding this to my .bashrc solved it: #write to history file at each shell prompt export PROMPT_COMMAND="history -a; history -c; history -r; $PROMPT_COMMAND" Every time a command is executed it's written to history right away. What you loose: the scroll-back buffer in each window pertaining to that session. So if you have a window running a set of commands, and you just 'up arrow' to re-run, you loose that because all history gets appended. I too would generally have 20+ open xterm windows but this gave instant command-to-history that I needed. fzf also helps instead of relying on up-arrow.
- mrweasel 4y ago
  I'm going to give this a go. That looks amazing, and it makes so much sense. I have a ton of commands around, either in my history, which isn't the best place, or in random files in different project folders. The ability to take a command and add to a notebook is a great idea.
- dewey 4y ago
  This looks great, one of these things I always wanted to build myself so I'm glad someone did a better job and shipped it! Thanks for sharing!
- pastage 4y ago
  Warning while this looks sweet, but this a security breach in the making. Having mistakes done in CLI local to the machine and session has saved me many times from storing really stupid stuff in the command history. I am not sure what safe guards are needed before I can use this. Which makes it hard to recommend it. Especially since security is not mentioned at all on that page.
  - kordlessagain 4y ago
    A good best practice is to always change tokens that get stored in history, no matter where that history is stored (which in this case is iCloud). IIRC, putting a space in front of the command will prevent it from appearing in history...
  - outcoldman 4y ago
    Just to clarify: 1. iCloud sync is optional. You can sync the history, or notebooks (or both). Or keep everything always local only. 2. ShellHistory syncs with your private iCloud account, nobody other than person having access to your AppleID (with 2nd factor auth) cannot get access to it. 3. I would say it could be even better than storing your history in a file. Considering that with the ShellHistory you have access to "Full-Text-Search" and can really quickly find any accidental leaked information. 4. In v2 I have implemented an ignore pattern with RegEx, where you can define which commands you want to be ignored from saving to database. 5. I am on purpose build the app sandboxed and distribute it via App Store. It declares what it does. I don't and would never implement any custom telemetry collection, other than what App Store provides me already.
zipperhead 4y ago
I'd given up on shared history because it never worked the way I like. This looks very promising, thank you.
skissane 4y ago
Storing history in SQLite is such a wonderful idea, I wonder why someone doesn’t just add it to readline/libedit/etc, then all apps (which use those libraries) can do it. Although, I wonder if that might cause problems, if apps already link to SQLite and they might expect a different version, or if unstable apps might cause concurrency/corruption/etc issues with a shared per-user history DB. To avoid all that, maybe a per-user history daemon with a Unix domain socket?
- pletnes 4y ago
  The xonsh shell can use sqlite for history. Really useful, also contains runtime statistics, for instance. I used xonsh for a few weeks. It is absolutely an awesome shell, but I didn’t find the switch worth the effort personally.
- zwp 4y ago
  Slightly silly sketch (bash): sqlite-utils create-table ~/commands.db commands id integer text text --pk id PROMPT_COMMAND="( fc -n -l -1 | perl -p -e 's/^\s+//; chomp if eof' | sqlite-utils insert --text ~/commands.db commands - & )" It's slow, has perl & python external deps, needs a timestamp column, call subshell to avoid job control messages, ... A nicer single "prompt command" wrapper is certainly possible though.

outcoldman 4y ago

If somebody is interested in macos app for storing your history (bash,zsh,fish) and syncing it with icloud, I have built an app ShellHistory, v2 is available via TestFlight if you want to try it. https://loshadki.app/blog/2022-09-22-public-beta-shellhistory-v2/ https://loshadki.app/blog/2022-09-22-public-beta-shellhistor... (monterey and ventura) And this is a link to v1 https://loshadki.app/shellhistory/ https://loshadki.app/shellhistory/ (big sur+)

InvaderFizz 4y ago
Been using this for about a year, well worth the $12. Thanks man! Saves me finding a command I ran but I don't remember when, or in which of my 50+ open and not committed shells. Also the exit code of every command is super helpful.
- outcoldman 4y ago
  Thank you for feedback! Appreciate to see a user from HN :D
- UI_at_80x24 4y ago
  >...or in which of my 50+ open and not committed shells... I suffered this frustration too, but adding this to my .bashrc solved it: #write to history file at each shell prompt export PROMPT_COMMAND="history -a; history -c; history -r; $PROMPT_COMMAND" Every time a command is executed it's written to history right away. What you loose: the scroll-back buffer in each window pertaining to that session. So if you have a window running a set of commands, and you just 'up arrow' to re-run, you loose that because all history gets appended. I too would generally have 20+ open xterm windows but this gave instant command-to-history that I needed. fzf also helps instead of relying on up-arrow.
mrweasel 4y ago
I'm going to give this a go. That looks amazing, and it makes so much sense. I have a ton of commands around, either in my history, which isn't the best place, or in random files in different project folders. The ability to take a command and add to a notebook is a great idea.
dewey 4y ago
This looks great, one of these things I always wanted to build myself so I'm glad someone did a better job and shipped it! Thanks for sharing!
pastage 4y ago
Warning while this looks sweet, but this a security breach in the making. Having mistakes done in CLI local to the machine and session has saved me many times from storing really stupid stuff in the command history. I am not sure what safe guards are needed before I can use this. Which makes it hard to recommend it. Especially since security is not mentioned at all on that page.
- kordlessagain 4y ago
  A good best practice is to always change tokens that get stored in history, no matter where that history is stored (which in this case is iCloud). IIRC, putting a space in front of the command will prevent it from appearing in history...
- outcoldman 4y ago
  Just to clarify: 1. iCloud sync is optional. You can sync the history, or notebooks (or both). Or keep everything always local only. 2. ShellHistory syncs with your private iCloud account, nobody other than person having access to your AppleID (with 2nd factor auth) cannot get access to it. 3. I would say it could be even better than storing your history in a file. Considering that with the ShellHistory you have access to "Full-Text-Search" and can really quickly find any accidental leaked information. 4. In v2 I have implemented an ignore pattern with RegEx, where you can define which commands you want to be ignored from saving to database. 5. I am on purpose build the app sandboxed and distribute it via App Store. It declares what it does. I don't and would never implement any custom telemetry collection, other than what App Store provides me already.

zipperhead 4y ago

I'd given up on shared history because it never worked the way I like. This looks very promising, thank you.

skissane 4y ago

Storing history in SQLite is such a wonderful idea, I wonder why someone doesn’t just add it to readline/libedit/etc, then all apps (which use those libraries) can do it. Although, I wonder if that might cause problems, if apps already link to SQLite and they might expect a different version, or if unstable apps might cause concurrency/corruption/etc issues with a shared per-user history DB. To avoid all that, maybe a per-user history daemon with a Unix domain socket?

pletnes 4y ago
The xonsh shell can use sqlite for history. Really useful, also contains runtime statistics, for instance. I used xonsh for a few weeks. It is absolutely an awesome shell, but I didn’t find the switch worth the effort personally.
zwp 4y ago
Slightly silly sketch (bash): sqlite-utils create-table ~/commands.db commands id integer text text --pk id PROMPT_COMMAND="( fc -n -l -1 | perl -p -e 's/^\s+//; chomp if eof' | sqlite-utils insert --text ~/commands.db commands - & )" It's slow, has perl & python external deps, needs a timestamp column, call subshell to avoid job control messages, ... A nicer single "prompt command" wrapper is certainly possible though.