3 ms·

Sorry to butt in, but this is a fallacy that I see expressed a lot (and shares a lot of similarities to the "computers can execute improperly-typed programs" fa

Sorry to butt in, but this is a fallacy that I see expressed a lot (and shares a lot of similarities to the "computers can execute improperly-typed programs" fallacy), and that has caused a non-trivial amount of harm. >> If digital, use only plain text. It’s a standard format not owned by any company. This is fine for anything where you don't care about the computer processing your data (e.g. a journal where you don't care about tagging or sorting by date or whatever), but it sounds like you do (based on "Based off of that structure I can do a bunch of cool things with simple scripts"), so: "Plain text" is mutually exclusive with machine-parsing, because "plain text" is not a format. If I create a text file for each of my friends, and somewhere in each file is an English-language description of my friend's name, then that is both (a) plain-text and (b) not machine-parseable at all. A "format" is necessarily structured - something like "each line of this file represents a "row", which is divided into "fields" by commas". But then you don't have a plain-text file - you have CSV, or JSON, or something that's a subset of "plain text file", but is still a structured, machine-parseable format. Even if you do what I did and create your own ad-hoc format like "text files are broken into "blobs", where each blob is separated from others by a blank line, and has a header consisting of the blob name..." - that's still not plain text. You've still invented your own structured-object-embedded-in-text format - except that because you aren't using JSON or XML or CSV, you're incompatible with every tool in existence, and have to write tools to parse, lint, generate, and process these files yourself. (and, hence, it's not a "standard format" - each ad-hoc plain-text-embedded format is different, except for the ones that are either accidentally exactly the same, or the ones that are standardized and have names - at which point plain-text purists claim that they're no longer plain-text) You use Markdown - which means that you have all of the problems above, because Markdown encodes formatting, not structure. You even alluded to it yourself - you say that "...I can do a bunch of cool things with simple scripts" because you had to write those scripts to handle your own custom format, instead of just using JSON and writing data["first-name"] in Python. It's your system, so you can do what you want - just don't fool yourself (or others) into believing that you're saving effort, because you're not - you're just reinventing a wheel that has been reinvented millions of times before. (this isn't meant to be a random rant - I'm currently in the middle of rewriting my own system from using my own custom, ad-hoc structured text format to using s-expressions and typed objects. there's hundreds of thousands of objects spanning thousands of files, and the process is absolutely miserable - I want to save as many other people from having to go through this as I can)

yardshop 5y ago
I think "plain text" in most cases really just means that it can be read and written by a human in a basic text editor. It is not mutually exclusive because parsers can be written to handle more freeform text than just csv and markdown. "Plain text" is like a carrier wave that can hold a format like CSV or markdown, or loosely conventioned prose that the right kind of parser can make reasonable guesses about. Your semi structured blob format and mine are still plain text. They are comprised of the basic set of characters from 32 to 126 maybe and 13 and 10, and maybe multi byte things these days. Plain text just ain't what it used to be anymore! They may not be standard formats, but they can be read and learned about in standard tools. But your cautions still ring true, as I have created several such formats and have yet to hit on something that is easy to write, and somewhat easy to process, and ticks all the other boxes. Good luck with your upgrade!
christiangenco 5y ago
That's fair. I suppose instead of "plain text" what I mean here is "minimally formatted plain text files that live on my hard drive."
skydhash 5y ago
I see your point. But as long as you're consistent with the format for each value, I don't foresee an issue. I consider everything that has a readable representation plain text, but your definition may be different. You can go with a standard format, like JSON or YAML, but I think you can probably define something that is easier to write and still processable. I agree with you, but someone can likely get by with adding markers and being consistent with them.

yardshop 5y ago

I think "plain text" in most cases really just means that it can be read and written by a human in a basic text editor. It is not mutually exclusive because parsers can be written to handle more freeform text than just csv and markdown. "Plain text" is like a carrier wave that can hold a format like CSV or markdown, or loosely conventioned prose that the right kind of parser can make reasonable guesses about. Your semi structured blob format and mine are still plain text. They are comprised of the basic set of characters from 32 to 126 maybe and 13 and 10, and maybe multi byte things these days. Plain text just ain't what it used to be anymore! They may not be standard formats, but they can be read and learned about in standard tools. But your cautions still ring true, as I have created several such formats and have yet to hit on something that is easy to write, and somewhat easy to process, and ticks all the other boxes. Good luck with your upgrade!

christiangenco 5y ago

That's fair. I suppose instead of "plain text" what I mean here is "minimally formatted plain text files that live on my hard drive."

skydhash 5y ago

I see your point. But as long as you're consistent with the format for each value, I don't foresee an issue. I consider everything that has a readable representation plain text, but your definition may be different. You can go with a standard format, like JSON or YAML, but I think you can probably define something that is easier to write and still processable. I agree with you, but someone can likely get by with adding markers and being consistent with them.