3 ms·

Hi Derek! My implementation of your system is just a bunch of markdown files. The title of the file is a friend's name, the frontmatter at the top keeps info a

Hi Derek! My implementation of your system is just a bunch of markdown files. The title of the file is a friend's name, the frontmatter at the top keeps info about them that doesn't change too often (ex: birthday, address, kids names), and each time I chat with them I add a section with an H1 title that's the date we chatted (ex: "# 2022-02-14\n"). My notes about the chat go under that heading. Based off of that structure I can do a bunch of cool things with simple scripts: - print out a list of upcoming birthdays, half birthdays, birthdays on Jupiter, etc. of all my friends - print out a list of friends I haven't contacted within my desired `frequency` (the subject of this post) - create or open the friend file containing a particular name from the command line (ex: `,people Sarah` prints out all the Sarah files I have and asks me which I'd like to open) I think plain text is the way to go with something as personal as a system for keeping track of your friends. I was inspired to go this route by your post on journaling[1]: > If digital, use only plain text. It’s a standard format not owned by any company. It will be readable in 50 years on devices we haven’t even imagined yet. Don’t use formats that can only be read by one program, because that program won’t be around in 50 years. Don’t use the cloud, unless you’re also going to download it weekly and back it up in plain text outside that cloud. (Companies shut down. Clouds disappear. Think long-term.) I've been chewing on how to make this system something that might be used by other people who don't know how to use the command line. Something like the way Obsidian[2] works might make sense. 1. https://sive.rs/dj https://sive.rs/dj 2. https://obsidian.md/ https://obsidian.md/

samatman 5y ago
Not at all to bikeshed your tool, which I'm sure works just fine, but this is the sort of application where GNU Recutils really shines. https://www.gnu.org/software/recutils/ https://www.gnu.org/software/recutils/ I mention it because it doesn't get the attention that it deserves, as a format midway between relational data in database and plain-old-text.
- sundarurfriend 5y ago
  The tooling (or lack thereof) is the only problem I have with recfiles. It quickly gets unpleasant entering the (repetitive) metadata every time (and the CLI entry isn't much better). I get the impression from the recfiles page that the Emacs plugin is the main supported one, and maybe that one has a good UI flow that makes it usable, but I don't use Emacs. I thought/hoped visidata would solve the problem. It has decent support for recfiles display - just being able to get a tabular view of the data is very useful - but the editing support is pretty unintuitive and full of footguns.
  - samatman 5y ago
    I'm glad you mentioned it because it's the format I would like to see get more attention, and I agree that tooling around it is holding it back. The repetition is genuinely useful if editing or adding one or two entries by hand, but tools for ingesting data into recfiles format, working with it in bulk and so on, I agree with you, they're primitive. My daydream is a tool like jq for recfiles, with something in front of it like pandoc for structured data, so that I can put my choice of JSON/TOML/YAML/TSV/CSV through the filter and work with it in recfile format. The real prize would be a solid library for importing and exporting SQLite data in recfiles form. The mapping is obvious and the metadata would allow for preservation of more of the underlying schema than the standard outputs, which are CSV and a collection of literal SQLite commands which you can execute to insert the data. I think pairing SQLite with a standard recfiles protocol would be exceptionally powerful, a workflow where SQLite reads a collection of recfiles into an in-memory database, slices and dices at leisure, and exports the changeset as patches over the recfiles? Yes please.
sivers 5y ago
That's amazing! In 25+ years of keeping people in my database, and writing plain text all day, I'd never once considered keeping my list of people in plain text files. Thanks for the description and inspiration.
- christiangenco 5y ago
  You bet! You're also one of my heroes so I'd be absolutely delighted to chat with you more about this problem on a call or something.
throw10920 5y ago
Sorry to butt in, but this is a fallacy that I see expressed a lot (and shares a lot of similarities to the "computers can execute improperly-typed programs" fallacy), and that has caused a non-trivial amount of harm. >> If digital, use only plain text. It’s a standard format not owned by any company. This is fine for anything where you don't care about the computer processing your data (e.g. a journal where you don't care about tagging or sorting by date or whatever), but it sounds like you do (based on "Based off of that structure I can do a bunch of cool things with simple scripts"), so: "Plain text" is mutually exclusive with machine-parsing, because "plain text" is not a format. If I create a text file for each of my friends, and somewhere in each file is an English-language description of my friend's name, then that is both (a) plain-text and (b) not machine-parseable at all. A "format" is necessarily structured - something like "each line of this file represents a "row", which is divided into "fields" by commas". But then you don't have a plain-text file - you have CSV, or JSON, or something that's a subset of "plain text file", but is still a structured, machine-parseable format. Even if you do what I did and create your own ad-hoc format like "text files are broken into "blobs", where each blob is separated from others by a blank line, and has a header consisting of the blob name..." - that's still not plain text. You've still invented your own structured-object-embedded-in-text format - except that because you aren't using JSON or XML or CSV, you're incompatible with every tool in existence, and have to write tools to parse, lint, generate, and process these files yourself. (and, hence, it's not a "standard format" - each ad-hoc plain-text-embedded format is different, except for the ones that are either accidentally exactly the same, or the ones that are standardized and have names - at which point plain-text purists claim that they're no longer plain-text) You use Markdown - which means that you have all of the problems above, because Markdown encodes formatting, not structure. You even alluded to it yourself - you say that "...I can do a bunch of cool things with simple scripts" because you had to write those scripts to handle your own custom format, instead of just using JSON and writing data["first-name"] in Python. It's your system, so you can do what you want - just don't fool yourself (or others) into believing that you're saving effort, because you're not - you're just reinventing a wheel that has been reinvented millions of times before. (this isn't meant to be a random rant - I'm currently in the middle of rewriting my own system from using my own custom, ad-hoc structured text format to using s-expressions and typed objects. there's hundreds of thousands of objects spanning thousands of files, and the process is absolutely miserable - I want to save as many other people from having to go through this as I can)

samatman 5y ago

Not at all to bikeshed your tool, which I'm sure works just fine, but this is the sort of application where GNU Recutils really shines. https://www.gnu.org/software/recutils/ https://www.gnu.org/software/recutils/ I mention it because it doesn't get the attention that it deserves, as a format midway between relational data in database and plain-old-text.

sundarurfriend 5y ago
The tooling (or lack thereof) is the only problem I have with recfiles. It quickly gets unpleasant entering the (repetitive) metadata every time (and the CLI entry isn't much better). I get the impression from the recfiles page that the Emacs plugin is the main supported one, and maybe that one has a good UI flow that makes it usable, but I don't use Emacs. I thought/hoped visidata would solve the problem. It has decent support for recfiles display - just being able to get a tabular view of the data is very useful - but the editing support is pretty unintuitive and full of footguns.
- samatman 5y ago
  I'm glad you mentioned it because it's the format I would like to see get more attention, and I agree that tooling around it is holding it back. The repetition is genuinely useful if editing or adding one or two entries by hand, but tools for ingesting data into recfiles format, working with it in bulk and so on, I agree with you, they're primitive. My daydream is a tool like jq for recfiles, with something in front of it like pandoc for structured data, so that I can put my choice of JSON/TOML/YAML/TSV/CSV through the filter and work with it in recfile format. The real prize would be a solid library for importing and exporting SQLite data in recfiles form. The mapping is obvious and the metadata would allow for preservation of more of the underlying schema than the standard outputs, which are CSV and a collection of literal SQLite commands which you can execute to insert the data. I think pairing SQLite with a standard recfiles protocol would be exceptionally powerful, a workflow where SQLite reads a collection of recfiles into an in-memory database, slices and dices at leisure, and exports the changeset as patches over the recfiles? Yes please.

sivers 5y ago

That's amazing! In 25+ years of keeping people in my database, and writing plain text all day, I'd never once considered keeping my list of people in plain text files. Thanks for the description and inspiration.

christiangenco 5y ago
You bet! You're also one of my heroes so I'd be absolutely delighted to chat with you more about this problem on a call or something.

throw10920 5y ago

Sorry to butt in, but this is a fallacy that I see expressed a lot (and shares a lot of similarities to the "computers can execute improperly-typed programs" fallacy), and that has caused a non-trivial amount of harm. >> If digital, use only plain text. It’s a standard format not owned by any company. This is fine for anything where you don't care about the computer processing your data (e.g. a journal where you don't care about tagging or sorting by date or whatever), but it sounds like you do (based on "Based off of that structure I can do a bunch of cool things with simple scripts"), so: "Plain text" is mutually exclusive with machine-parsing, because "plain text" is not a format. If I create a text file for each of my friends, and somewhere in each file is an English-language description of my friend's name, then that is both (a) plain-text and (b) not machine-parseable at all. A "format" is necessarily structured - something like "each line of this file represents a "row", which is divided into "fields" by commas". But then you don't have a plain-text file - you have CSV, or JSON, or something that's a subset of "plain text file", but is still a structured, machine-parseable format. Even if you do what I did and create your own ad-hoc format like "text files are broken into "blobs", where each blob is separated from others by a blank line, and has a header consisting of the blob name..." - that's still not plain text. You've still invented your own structured-object-embedded-in-text format - except that because you aren't using JSON or XML or CSV, you're incompatible with every tool in existence, and have to write tools to parse, lint, generate, and process these files yourself. (and, hence, it's not a "standard format" - each ad-hoc plain-text-embedded format is different, except for the ones that are either accidentally exactly the same, or the ones that are standardized and have names - at which point plain-text purists claim that they're no longer plain-text) You use Markdown - which means that you have all of the problems above, because Markdown encodes formatting, not structure. You even alluded to it yourself - you say that "...I can do a bunch of cool things with simple scripts" because you had to write those scripts to handle your own custom format, instead of just using JSON and writing data["first-name"] in Python. It's your system, so you can do what you want - just don't fool yourself (or others) into believing that you're saving effort, because you're not - you're just reinventing a wheel that has been reinvented millions of times before. (this isn't meant to be a random rant - I'm currently in the middle of rewriting my own system from using my own custom, ad-hoc structured text format to using s-expressions and typed objects. there's hundreds of thousands of objects spanning thousands of files, and the process is absolutely miserable - I want to save as many other people from having to go through this as I can)