5 ms·

Because this comes up every so often, I'll post this link again: I wrote a little widget that stores your bash history in a sqlite db. I haven't touched it in a

Because this comes up every so often, I'll post this link again: I wrote a little widget that stores your bash history in a sqlite db. I haven't touched it in a while because I now rely on fzf and this doesn't integrate with it, though maybe I could make it do... https://github.com/thenewwazoo/bash-history-sqlite https://github.com/thenewwazoo/bash-history-sqlite

chubot 7y ago
This is cool, somebody recently suggested that I add history to sqlite in Oil [1]. I looked at your code but I didn't see how it integrates with bash? Most people seem to be using the PROMPT_COMMAND hook for stuff like this -- are you using something else, or did I miss it? https://github.com/oilshell/oil/issues/320 https://github.com/oilshell/oil/issues/320 I will add PROMPT_COMMAND to Oil, but I'm curious if there is something else I should add to support end-user customization. One thing that might be interesting is to support a hook for the ! syntax, so the sqlite DB could be searched. I think that would solve some of the problems you mention in the code's comments, and maybe the fzf issue. (Although that literally just occurred to me, I don't promise to do this :) ) I haven't looked at how fzf works exactly. But if you want to chat about it please chime in on Github :) [1] Latest status: Success With the Interactive Shell http://www.oilshell.org/blog/2019/02/05.html http://www.oilshell.org/blog/2019/02/05.html . Oil runs a bunch of interactive programs like bash-completion, virtualenv, and git-prompt!
- skissane 7y ago
  > This is cool, somebody recently suggested that I add history to sqlite in Oil I think a shell with an embedded sqlite, which makes sqlite accessible to scripts as shell builtins, and in which command line history is stored in a sqlite database instead of a plain text file, would be pretty cool. You could store command history (and maybe even other stuff) in something like $HOME/.shell_db As well as just a list of lines, other useful information could be stored for each line, such as current directory, timestamp, username (maybe I su to root and am running commands as root but still going into my own account's history database), hostname (if $HOME is on NFS, putting aside the issue that databases, including sqlite, don't work well over NFS due to file locking issues), etc.
- enriquto 7y ago
  > somebody recently suggested that I add history to sqlite in Oil what a disgusting, perturbed mind! Shell history is probably the kind of data that fits better as a text file. It is naturally textual and it can never grow too long, even if you are typing commands continuously for your entire life. Please, keep the dirty hands of sql far, far away from the shell!
  - stevekemp 7y ago
    I can almost see the appeal, when you think of decorating it. Consider a table with five columns: * uid * pwd * cmd * exit-code * datetime Now you can `SELECT pwd,cmd WHERE exit-code != 0` to see all failed-commands.
    - enriquto 7y ago
      I see your "decorated" sql table, and I raise you my text file with five columns, that you can grep and sed and awk as if there was no tomorrow. cat history.txt | awk '$4 != 0'
      - zepolen 7y ago
        "As if there was no tomorrow" - exactly, because once you add another column suddenly $4 points to something completely different and you have to rewrite everything. Not to mention $4 is completely arbitrary and it's impossible to tell without prior knowledge which field it's referencing. The original query `SELECT pwd, cmd WHERE exit-code != 0` is both descriptive and won't break with future updates.
    - chubot 7y ago
      Yeah I agree. In Oil I'm hoping to have the best of both worlds. That is, no silly quoting and parsing of strings, but also being able to use plain tools like "grep", and probably SQL. https://github.com/oilshell/oil/wiki/Structured-Data-in-Oil https://github.com/oilshell/oil/wiki/Structured-Data-in-Oil Note that R has some popular packages that allow SQL syntax over data frames. So basically the history file could be a textual table. You could trivially import it into a database, or you can just grep it.

chubot 7y ago

This is cool, somebody recently suggested that I add history to sqlite in Oil [1]. I looked at your code but I didn't see how it integrates with bash? Most people seem to be using the PROMPT_COMMAND hook for stuff like this -- are you using something else, or did I miss it? https://github.com/oilshell/oil/issues/320 https://github.com/oilshell/oil/issues/320 I will add PROMPT_COMMAND to Oil, but I'm curious if there is something else I should add to support end-user customization. One thing that might be interesting is to support a hook for the ! syntax, so the sqlite DB could be searched. I think that would solve some of the problems you mention in the code's comments, and maybe the fzf issue. (Although that literally just occurred to me, I don't promise to do this :) ) I haven't looked at how fzf works exactly. But if you want to chat about it please chime in on Github :) [1] Latest status: Success With the Interactive Shell http://www.oilshell.org/blog/2019/02/05.html http://www.oilshell.org/blog/2019/02/05.html . Oil runs a bunch of interactive programs like bash-completion, virtualenv, and git-prompt!

skissane 7y ago
> This is cool, somebody recently suggested that I add history to sqlite in Oil I think a shell with an embedded sqlite, which makes sqlite accessible to scripts as shell builtins, and in which command line history is stored in a sqlite database instead of a plain text file, would be pretty cool. You could store command history (and maybe even other stuff) in something like $HOME/.shell_db As well as just a list of lines, other useful information could be stored for each line, such as current directory, timestamp, username (maybe I su to root and am running commands as root but still going into my own account's history database), hostname (if $HOME is on NFS, putting aside the issue that databases, including sqlite, don't work well over NFS due to file locking issues), etc.
enriquto 7y ago
> somebody recently suggested that I add history to sqlite in Oil what a disgusting, perturbed mind! Shell history is probably the kind of data that fits better as a text file. It is naturally textual and it can never grow too long, even if you are typing commands continuously for your entire life. Please, keep the dirty hands of sql far, far away from the shell!
- stevekemp 7y ago
  I can almost see the appeal, when you think of decorating it. Consider a table with five columns: * uid * pwd * cmd * exit-code * datetime Now you can `SELECT pwd,cmd WHERE exit-code != 0` to see all failed-commands.
  - enriquto 7y ago
    I see your "decorated" sql table, and I raise you my text file with five columns, that you can grep and sed and awk as if there was no tomorrow. cat history.txt | awk '$4 != 0'
    - zepolen 7y ago
      "As if there was no tomorrow" - exactly, because once you add another column suddenly $4 points to something completely different and you have to rewrite everything. Not to mention $4 is completely arbitrary and it's impossible to tell without prior knowledge which field it's referencing. The original query `SELECT pwd, cmd WHERE exit-code != 0` is both descriptive and won't break with future updates.
  - chubot 7y ago
    Yeah I agree. In Oil I'm hoping to have the best of both worlds. That is, no silly quoting and parsing of strings, but also being able to use plain tools like "grep", and probably SQL. https://github.com/oilshell/oil/wiki/Structured-Data-in-Oil https://github.com/oilshell/oil/wiki/Structured-Data-in-Oil Note that R has some popular packages that allow SQL syntax over data frames. So basically the history file could be a textual table. You could trivially import it into a database, or you can just grep it.