3 ms·

The tooling (or lack thereof) is the only problem I have with recfiles. It quickly gets unpleasant entering the (repetitive) metadata every time (and the CLI en

The tooling (or lack thereof) is the only problem I have with recfiles. It quickly gets unpleasant entering the (repetitive) metadata every time (and the CLI entry isn't much better). I get the impression from the recfiles page that the Emacs plugin is the main supported one, and maybe that one has a good UI flow that makes it usable, but I don't use Emacs. I thought/hoped visidata would solve the problem. It has decent support for recfiles display - just being able to get a tabular view of the data is very useful - but the editing support is pretty unintuitive and full of footguns.

samatman 5y ago
I'm glad you mentioned it because it's the format I would like to see get more attention, and I agree that tooling around it is holding it back. The repetition is genuinely useful if editing or adding one or two entries by hand, but tools for ingesting data into recfiles format, working with it in bulk and so on, I agree with you, they're primitive. My daydream is a tool like jq for recfiles, with something in front of it like pandoc for structured data, so that I can put my choice of JSON/TOML/YAML/TSV/CSV through the filter and work with it in recfile format. The real prize would be a solid library for importing and exporting SQLite data in recfiles form. The mapping is obvious and the metadata would allow for preservation of more of the underlying schema than the standard outputs, which are CSV and a collection of literal SQLite commands which you can execute to insert the data. I think pairing SQLite with a standard recfiles protocol would be exceptionally powerful, a workflow where SQLite reads a collection of recfiles into an in-memory database, slices and dices at leisure, and exports the changeset as patches over the recfiles? Yes please.

3 ms·

The tooling (or lack thereof) is the only problem I have with recfiles. It quickly gets unpleasant entering the (repetitive) metadata every time (and the CLI en

by sundarurfriend 5y ago

samatman 5y ago
I'm glad you mentioned it because it's the format I would like to see get more attention, and I agree that tooling around it is holding it back. The repetition is genuinely useful if editing or adding one or two entries by hand, but tools for ingesting data into recfiles format, working with it in bulk and so on, I agree with you, they're primitive. My daydream is a tool like jq for recfiles, with something in front of it like pandoc for structured data, so that I can put my choice of JSON/TOML/YAML/TSV/CSV through the filter and work with it in recfile format. The real prize would be a solid library for importing and exporting SQLite data in recfiles form. The mapping is obvious and the metadata would allow for preservation of more of the underlying schema than the standard outputs, which are CSV and a collection of literal SQLite commands which you can execute to insert the data. I think pairing SQLite with a standard recfiles protocol would be exceptionally powerful, a workflow where SQLite reads a collection of recfiles into an in-memory database, slices and dices at leisure, and exports the changeset as patches over the recfiles? Yes please.