13 ms·

If it is worth keeping, save it in Markdown

• gnuser 2y ago
  I suggest emacs org mode or asciidoc
• aosaigh 2y ago
  I'm not surprised this post opens with a link to /r/DataHoarder. Hot take ... I understand the sentiment that you can't trust content on the web to be there forever, but there is also the other side of the argument which is: compulsively saving data is a waste of time and it introduces a cognitive overhead that you'd be better off without.
  • atrus 2y ago
    I'd disagree a bit there. I do something similar, saving interesting webpages, and it's really really nice being able to quickly search for something that I halfway remember a few months down the line. I'm not saving everything, and it just gets stuffed unedited into a folder that I can search. Not too much in the cognitive overhead department.
    • turtlebits 2y ago
      Until run you out of space and/or storage bill is too expensive.
      • packetlost 2y ago
        if you're storing text that is realistically not going to be a problem.
  • packetlost 2y ago
    > If it's worth saving Idk, I think if it's worth saving it's worth saving and the only person who can determine if it's "worth it" is me. I agree that some people have an obsession where they save data that isn't worth it, but r/DataHoarder is a great place with a lot of information on building and maintaining large data systems for hobbyists, regardless of what you actually store.
    • xpe 2y ago
      Actually, “future you” will know even better in many cases. Also, some people other than yourself might turn out to be a better predictor of “future you” than “current you”. So many people are hyper individualistic and forget just how predictable individuals can be to others!
  • theshackleford 2y ago
    > I understand the sentiment that you can't trust content on the web to be there forever The thing is, people say this, and I am sure for some amount of content it's true. However I eventually realised I have never had a single issue if required, in retrieving literally any piece of software or digital content after the fact from somewhere on the internet. It's pretty much why I care so little about what happens to my steam library when Gaben kicks it. If I get the urge to replay something in twenty years that I paid $3 dollars for and its suddenly gone, i'll just go find it elsewhere. Once I had gotten pretty much everything I wanted from running a large scale storage system (largely to learn the in's and outs of linux/general storage concepts) I pretty much just gave it up. Its a lot of money to hold onto things that at this point, I pretty much know i'll always be able to recover elsewhere. I'd rather someone else pay the electricity/drive cost for me.
    • weikju 2y ago
      That’s only possible because some people are hoarders who save this data or software and put it out there..
      • theshackleford 2y ago
        > That’s only possible because some people are hoarders who save this data or software and put it out there.. Sure. And I appreciate it. Like I appreciate the internet archive etc.
        • johnfernow 2y ago
          There is no comparable alternative to the Internet Archive though. They've gotten involved in several lawsuits and their future is far from guaranteed. They're an incredibly important organization, but I think it's too important of a project to be limited to one organization, or even one country or region of the earth. A solar flare could destroy a lot of history. I don't know that the economics of having multiple Internet Archive-like organizations is currently feasible (I imagine getting funding for one of them is hard enough), but even a partial offline mirror hosted someplace else would be nice. Maybe to save space they could take the oldest version of a page, the newest, and the midmost version timewise, discarding all other versions. They could also heavily compress images, video and audio to save storage space (would increase processing costs, but if willing to throw out quality, could compress quickly and still save a bunch of space. E.g. downscale all videos to 480p and use veryfast preset and CRF 28 with ffmpeg. Even 240p is a lot better than nothing. A pixelated form of history is better than no history.)
  • NeutralForest 2y ago
    When I find a blog post/article I'm interested in, I save it to my laptop with the SingleFile extension and I take quick notes as well I write my thoughts about it in org-mode. It has a very low cognitive threshold and I can always read it back in the browser. I'm find if not all the outbound links are still working, I'd just like to read back sometimes.
    • DaSHacka 2y ago
      This is almost exactly what I do too, though I also throw the link in the wayback machine so it's easier to share with others should the source go down (and to be courteous to any like-minded fellows who also wanted to see the content, but unfortunately came too late)
  • filcuk 2y ago
    I agree, but I also think that data hoarding is similar to regular hoarding. All of this content and information seems like it could be useful, invaluable even. It's a problem in a world where we have excess to sort through all that information and only focus on what's important right now.
  • maxbond 2y ago
    An archivist saves everything, because it's impossible to anticipate what future historians will need or want; they are working from a context you cannot access. You can winnow down a large dataset to it's relevant subset, but you can't study what wasn't preserved.
  • polytely 2y ago
    If I read something and then remember it 5 years later because it becomes relevant, I want to be able to find it. It's not even that I will look at it, I just want to have the option if I want to.
  • euroderf 2y ago
    Nobody is taking their data hoard and feeding it into a personal AI ?
• headcanon 2y ago
  I've been doing this recently with every URL I've bookmarked over the last 15 years or so since I signed up for pinboard.in. http://spider.cloud http://spider.cloud has been really nice for crawling sites and saving the results as markdown. I plan on expanding it to transcribing youtube videos I've saved, github repos I've starred, HN posts, etc. Ultimately I'm trying to index my "window" to the web as embedded content in a vector store. Not sure exactly what I'm going to do with it yet but I imagine it will be a component of some kind of personal agent system I can use to reference old info and help as a writing tool or as an "idea generator" of some kind. I'll likely end up not using most of it but you never know. I've scraped about 10k markdown files which has created a ~10gb chromadb instance so far. Eventually I'll probably create separate collections based on domain, and filter down items that I care about more.
• normalaccess 2y ago
  I love markdown and use it for all my notes, however it really needs a native way to underline. I have been converting some older books and lectures to markdown and underline is used all the time. If anyone has a good solution I'm all ears.
  • grimui 2y ago
    Not the best solution but you could use the HTML underline tag
  • ambivalence 2y ago
    Markdown is plaintext so you decide what it means. I personally write *italic* and **bold**, so I can use _underline_. Most Markdown to HTML converters would make the last example into italic, but you can customize many of them. Commonmark doesn't even mention "bold", "italic", and "underline". It just says "emphasis" and "strong emphasis". You can style it however you want.
    • paulryanrogers 2y ago
      This kind of undercuts the advantage of a semi-universal format. Though I'd agree underscore wrappers are quite reasonable and natural.
      • ratorx 2y ago
        Markdown isn’t really meant to be a universal markup format. Its primary goal is to document conventions of annotating plain text which keep the plaintext semi-consistent and readable. So the purpose of , * etc is purely emphasis. If you need to represent something specific (bold, italic etc) then that’s a job for the Markdown parser (or embedded HTML etc). The result of the parser (HTML, etc) will be less human readable, but actually able to specify formatting. I agree that CommonMark could be extended, but I think the focus should be on semantic* relevance rather than markup specification.
  • Tagbert 2y ago
    it might depend on what you want to do with the underline. Does it just indicate some kind of emphasis? Could you use the convention in your documents that "_" is the underline delimiter? I know that the default is to render it as italic/emphasis but that is just a decision at rendering time. The semantics of emphasize/underline could easily overlap. Of course if you want 3 levels of emphasis with bold, italic, and underline, then yes you need to look elsewhere. Markdown isn't really a formatting tool. it is a way to structure text in the minimal way that a person would interpret it and a machine could render it.
• ambivalence 2y ago
  I wholeheartedly agree with this post. I also keep my notes in Markdown, I also have plenty of Python scripting around them, including automatic publishing of my website. I use FSNotes today on macOS and iOS. Both apps are open source, both use well-structured .textbundle directories that separate Markdown content from JSON metadata and binary attachments. Synchronization happens through Git. It's a very powerful combination. Ironically, I wrote a blog post some 8 years ago about this very subject. That blog post is now offline.
  • ydant 2y ago
    I appreciate the mention of FSNotes (and in turn textbundle). Somehow, despite trying tons of note taking apps and formats, I don't remember ever coming across mention of this format specifically. http://textbundle.org/ http://textbundle.org/ My biggest beef with org mode and all of the markdown apps I've tried is the asset management problem. For me screenshots are almost as important as the text part of the note, and are usually strongly tied to a single note. I've taken to using apple notes at work just because it "solves" that well enough, but I'd really prefer to work in markdown/plain text (except for the images).
• Evidlo 2y ago
  Just a note that the most common Markdown flavor (Commonmark) doesn't actually support frontmatter. The author is using presumably Obisidian-flavored Markdown (which is a mixture of Commonmark, GH-flavored Markdown, and Latex). For file-tagging, I would consider TMSU [0] instead of writing bespoke tools. (ideally we would just use xattrs, but the world isn't ready for that) [0]: https://tmsu.org/ https://tmsu.org/
  • jorvi 2y ago
    I like Commonmark but I wish it would have been more opinionated. They chose to allow two ways to do everything[0]. Making * always be used for bold and _ always for italicizing is so much clearer, and some Markdown flavors (notably WhatsApp) do this. So you only have to do *haha* or _haha_, which also makes italic-bold more _*intuitive*_. Similarly they should have gone with one style of headings, probably with #. This frees up more visual clarity. Because you are no longer using *** for bold-italic, you can use that for lines, instead of both --- and ***. This then further frees --- up to be used for tables. Although I imagine there's a decent subset of people that uses the alternate style of doing headings === and the 'normal' way of doing lines ---, which would have killed adoption. And good luck convincing people to adopt a new variant at this point. "Commonermark"? "Peasantmark"? "Rabblemark" actually sounds decent. Edit: actually, having checked the discourse around it a bit more, Commonmark wasn't created as "one Markdown to rule them all", but rather as "Venn diagram markdown with the most overlap". [0]https://commonmark.org/help/ https://commonmark.org/help/
    • Shog9 2y ago
      If you want strictness, use a linter or a pretty-printer that follows your preferred style. Adopting an opinionated parser means you can't lint or pretty-print input from those with different opinions (I do not like underscores for emphasis), and thus somewhat goes against the goals of TFA here: > Markdown files are essentially plaintext with some extra syntax for common elements like sections, bullet points, and links. The format deliberately avoids precise control over display details like font selection. Following the rule of least power, I consider this limitation a feature. One of my biggest ongoing frustrations has been MDX - a sort of markdown-and-JSX mixture whose spec is now in its third release and which has made very little effort to maintain compatibility with either CommonMark or itself. It is fairly strict and fairly elegant, and moving to a new version requires rewriting all previously-written documents to eliminate no-longer-supported syntax and re-training writers. Both of those things are miserable tasks; it has absolutely killed any tolerance I might have had for a stricter parser.
• downut 2y ago
  The underlying purpose of org-mode is to manage this issue (the text part). It doesn't solve it, instead it is a tool for managing the steadily increasing archive organizational complexity within an ever evolving timeline. You reconfigure your archive's implicit schema well now you're in a world of heavy editing. That's life. If you don't have a solid backup strategy, you are going to lose stuff. That's also life. Big binary blobs are a different, equally important problem. Sure, keep your archive text in markdown (which one? a dumb person asks). But I'd recommend managing it with org-mode, it doesn't really care what format your text is in. (Yeah I saw the footnote mentioning org-mode but that reads to me that org-mode's reference there is entirely about the markup flavor.)
  • aquariusDue 2y ago
    Yeah, org-mode and by extension Emacs really help in this regard. Now that Emacs has been ported to Android I expect its usefulness to only increase. Looking back I can't believe I considered just bookmarking a link enough to save it long-term. Sure, I lost a lot of cruft but there were some gems that in retrospect I'd have liked to still reference or look at today. Eh, hindsight is 20/20 as the saying goes.
• roxolotl 2y ago
  I’ve been self hosting linkding[0] and it has archiving capabilities. Saves in html not markdown but that’s basically the same thing. It’s been very useful and then I back the folder up to R2 for free. I enjoy knowing that if I find something I want to remember it won’t go away. Plus it works great for recipe sites because I don’t have to deal with ads. [0]: https://github.com/sissbruecker/linkding https://github.com/sissbruecker/linkding
  • youainti 2y ago
    I love linkding. I try to add sites to it whenever I'm tempted to just keep the tab open for later because it is cool.
• asielen 2y ago
  100% agree. I've been using markdown for a few years after moving away from proprietary note taking apps. Although this has led to me developing my own short hand for many things in my notes. And have been looking at a way to integrate a to-do list with my notes with some Python scripts. So while my notes may rely on some personal scripts to get there most value out of them, I strongly value that they are still plain text and I can always move them into a new workflow if I need to.
  • Onawa 2y ago
    Maybe consider Quarto? Free and open source, integrates Python scripting directly in Markdown and/or able to call from separate scripts.
    • darkteflon 2y ago
      Love Quarto. I write all my notes, presentations, blog posts, memos, etc in .qmd files. For non-technical stuff I use Obsidian to author (there is an extension which tells Obsidian to treat .qmd as ordinary markdown - ie ignoring the additional Quarto frontmatter and so on), then for everything else I use VS Code with the Quarto extension and just render out to the display format I need. I really appreciate that it’s built on Pandoc and it means I can just use one format and one set of tooling for everything.
    • asielen 2y ago
      Thanks for the recommendation! Sounds interesting
• hartator 2y ago
  The only drawback of Markdown is images. GitHub-flavored Markdown is so popular because you can really easy inline them. You don't have to worry about storing them, linking them correctly, and you can even paste to the Markdown field. There is no elegant solution like this in actual Markdown.
  • darkteflon 2y ago
    I would add tables to that. Obsidian has some nice extensions to make working with tables easier, but it always feels janky.
    • hartator 2y ago
      Yeah, what people like about Markdown is basically regular text. The formatting is interesting, but it’s not revolutionary or anything.
    • josephg 2y ago
      Markdown is also missing: - Diagrams - Math - Any custom blocks - like Figures, algorithms, image boxes, etc - Numbered chapters / sections (Eg Chapter 1, Appendix A, etc). - Semantic references For readmes, thats fine I guess. But I miss all those features when I'm writing proper blog posts, articles and documentation. There are various hacky workarounds. But as soon as you start using bespoke markdown extensions, you're locked out of 95% of the markdown tooling out there. And everything feels so janky. I'm looking forward to Typst's HTML output getting more mature. Typst is the only typesetting tool I've ever used that is both enjoyable to use and powerful enough for the kind of documents I want to write. It manages that by being a full on programming language. You can define variables and write custom functions for reused blocks. And there is an ecosystem of 3rd party typst packages. For a paper I wrote recently, my benchmarking tool spat the results into a JSON file. My document loaded that JSON data directly, and used the benchmarking results to populate charts and tables in the paper. It was crazy cool.
      • Refusing23 2y ago
        i do believe obsidian support all those (well maybe not all) but thats added 'on top' of basic markdown, so when you open your markdown file in some other program it looks weird
• paulryanrogers 2y ago
  My favorite is WikiCreole, with (subset of) HTML as a close second. MD is alright, but too restrictive as a general purpose format for knowledge bases and such.
  • xpe 2y ago
    Right. Markdown ain’t the only game in town.
• huqedato 2y ago
  My pain is that I couldn't find a decent md viewer for Windows: free, fast, simple, no distractions. Imagine notepad. I have to open my md files with VSCode or Notepad++ (nasty view).
  • 2OEH8eoCRo0 2y ago
    I have a small wrapper for pandoc with sane defaults to create PDFs of my notes which I can print out and write on. Some of my scribblings are useful or important enough to get added to the markdown files and printed the next time.
  • florbnit 2y ago
    Easy fix, just write your md files in HTML, that way they are easy to read, and when you need to look at the markdown you can just use a converter.
    • DidYaWipe 2y ago
      That's sarcasm, right?
  • dontdieych 2y ago
    Browser extension as MD reader is quite nice. - https://markdownreader.github.io https://markdownreader.github.io - https://github.com/simov/markdown-viewer https://github.com/simov/markdown-viewer Those two are quite good.
  • dodorz 2y ago
    Try this one: Markdown Viewer. Other than the larger size that comes with being based on Electron, it's all quite good. https://github.com/c3er/mdview https://github.com/c3er/mdview
    • huqedato 2y ago
      Thanks. It looks pretty decent. And more important is the first one being just a reader/viewer not an editor with preview.
• ThinkBeat 2y ago
  When it comes to text (though I do include Word, PDF, text files, markdown, tex) I like to burn them to dvd. I have one of those big dvd "catalogs" that takes 4 discs per side of a page. Keep one at home and one at my parents' place. I trust them more than usb-sticks. Though that may be irrational. But the time for burning files to dvd seems almost over. It is hard /impossible to buy a computer with a dvd drive. That is no problem for me since I have a collection fo externals as well as internals. and life is good now since blank dvd media is cheap . But again, you need a dvd reader, and in the future, that may become difficult.
  • tpmoney 2y ago
    The trick with burning optical media is the disks themselves can physically fail with time. I have a huge archive of various burned media from the early 00's and a number of them have developed literal holes in the material over the years. If these holes hit data tracks, the files on those tracks are lost. If you're burning to optical media, you should probably be checking them regularly for degradation.
    • ThinkBeat 2y ago
      This is true. And a real problem. In my experience hard drives, USB sticks fail and regular hard drives fail. It has been many years since I have had any involvement but backup tapes probably have issues as well, but the rapid production of new tapes and new formats is an issue already I dont have any data to evaluate the best choice is SSD drives? No matter what technology is picked, at some point to preserve the data it needs to be migrated to whatever comes down the line.
      • tpmoney 2y ago
        Honestly, so far if you can afford the up-front investment for the space you need, and the ongoing power costs, a NAS with a RAID array (or similar redundancy scheme) that can tolerate more than one drive failure at a time is probably the best long term archival storage. Spinning rust disks in my experience rarely completely fail without warning so you can usually catch and replace failing media before data loss occurs. Additionally if you don't, I've found that recovering data from failed HDD is also usually "easier" and "cheaper" for most values of both compared to other media storage (admittedly with no experience with recovering tape media)
• jon_richards 2y ago
  The killer app for markdown would be a collaborative editor that displays the raw markdown and formatted markdown side-by-side and makes both sides editable. Tech people can use `#` and `*` on one side for formatting, product people can use normal text-editor buttons like "header1", "italics", etc.
  • jxjnskkzxxhx 2y ago
    Sounds like an obsidian plugin.
    • mock-possum 2y ago
      Notepad++ and VSC both have plugins for this. They’re fine.
    • philips 2y ago
      Someone is building that: https://screen.garden https://screen.garden
  • r3trohack3r 2y ago
    I built this in college, but the code is lost. It was a week or so of hacking. I believe in you. IIRC the trick was to get a pipeline for Markdown to HTML, render it into a WYSIWYG editor, then convert the HTML to an AST, and walk that to generate the markdown. I had to “format” both the markdown and html on each render (bidirectional round trip render) because parsing/gen wasn’t whitespace stable.
  • jkingsman 2y ago
    It's not collaborative, but this is what I love about Typora[0]. Click into a styled area and the styling becomes visible. Click out, and you just see the final styling. [0] https://typora.io/ https://typora.io/
    • Helmut10001 2y ago
      Typora is really a gem in the Markdown universe.
• k__ 2y ago
  AsciiDoc is basically DocBook-Markdown, which makes it a medium-independent format.
• kjs3 2y ago
  We never should have stopped using troff.
• tengwar2 2y ago
  The other major alternative to consider is RTF. I standardised on that about 10y ago, planning for a 30y horizon. It is a more complex format than Markdown, still text-based, but biased towards WYSIWYG presentation and editing, while Markdown is usually not WYSIWYG in the editor. Both formats suffer from a lack of standardisation, though Markdown seems to have more problems in practice - I've never had an issue caused by RTF incompatibility. Both are very widely supported. Both formats are very widely supported and it can reasonably be expected that this will continue. I prefer RTF for two main reasons: * I can't express simple formatting such as "make this text red" in Markdown. No, I don't mean "accentuate this text and leave the decision on how it looks to someone else", I really do mean "make this text red". I do a lot of public speaking, and I want to keep to certain conventions which are easy to read fast. * Most of the time I am writing text, not reading a version after it goes through a formatter, so I prefer to see it formatted on screen. That's really a limitation on Markdown editors, but it's almost universal so for my point of view, it counts.
  • caseyy 2y ago
    Markdown editors like Typora somewhat close this gap in functionality.
  • 0x073 2y ago
    * Embedded images
    • e2021 2y ago
      markdown can do embedded images with data urls. It works really well, the data gets all appended at the end of the document
  • nicoburns 2y ago
    HTML is also worth considering. It's not quite as nice to write in as Markdown, but it's still pretty reasonable to hand write (or even read if it comes to it).
    • copperx 2y ago
• kreelman 2y ago
  Hmmm. I see the use in this... For me, everything swirls in a lovely vortex towards org-mode. - Literate Programming, tangel/weave - Export to DocX, PDF, HTML - Org-Roam - Time Management.
• jazz9k 2y ago
  Obsidian is the killer app for this. I spent a month converting around 3 years of security notes to markdown and now use obsidian to search/archive everything.
• t_mann 2y ago
  Can relate to that sentiment. What I'm still looking for is a simple solution that lets me use simple local files (eg plaintext/markdown; csv or single-page HTML would also be fine) as a backend for a web app (with login, obviously). Basically, I want to have something like a todo.txt that lives on my machine (in the folder that syncs to my cloud storage) but that I can also edit when I'm on my phone. Like using Google sheets as a backend but with a local file.
  • vadansky 2y ago
    I just access my markdown files from Obsidian through nextcloud. When I'm on my phone I just use a simple markdown editor, when I'm on my PC I use Obsidian.
    • t_mann 2y ago
      Do you use any plug-ins for that? Obsidian tells me it only supports Obsidian Sync and iCloud out of the box.
      • sarnu 2y ago
        You don't have to use any plugins. You can put your obsidian vault anywhere you like, e.g. in a folder that is synched by nextcloud. I use a git repo for this, which works fine also on mobile.
        • t_mann 2y ago
          I get how that works on desktop, but on mobile, I can add a local file as a vault in Obsidian, but I don't think that file could be tracked by my cloud sync app. Does the Nextcloud app support that? Not sure how you use git here, could you explain? What I have gotten to work was to download a file from the sync app, open it in a markdown editor app and then save it to the cloud by sending it back to the sync app. It technically works but it was a bit too inconvenient to become a real habit (too many taps, need to rename the file on upload and set location each time,...).
  • 2y ago
• croes 2y ago
  Why not Asciidoc instead of Markdown?
  • kstrauser 2y ago
    AsciiDoc's fine. So is reStructuredText. In some ways they're both a lot better than Markdown, even though I think MD's surely easier to learn and use. But the one clear advantage MD has over the others is its ubiquity. If a tool works with formatted text, it almost certainly supports MD. It might also support the others, but if so, that's just a bonus.
    • euroderf 2y ago
      I get the feeling that the next step in this evolution is Typst.
• runevault 2y ago
  I actually use a VS Code plugin for this called Dendron. It is in the same vein as Obsidian or Notion, markdown based, and just runs in VSC. Very handy and since plain text works wonderfully in a git repository.
• misterspaceman 2y ago
  I've landed on a workflow that I like a lot, and have shown to several people on my team. I use Google Drive for Desktop, which maps the G:\ drive to Google Drive. From there, I use VS Code for Markdown editing. Google Docs now supports Markdown files, so if I need to convert the Markdown file to Word or PDF, I just open it in Docs and download it in the format I need. (Pandoc also works for this, as the author mentions). Converting HTML to Markdown can also be done in Docs: copy and paste the web page text into Google Docs, and download the file as Markdown. For mobile, I use the DriveSync app to download my notes (Markdown) folder to my phone. Then I use Obsidian to open and edit the files.
  • DidYaWipe 2y ago
    But why should you need to convert it? Why does no one call out the giant problem with Markdown: the lack of READERS?
    • immibis 2y ago
      People read documents, not formats.
      • DidYaWipe 2y ago
        Exactly. So why are we supposed to read Markdown full of embedded codes?
    • misterspaceman 2y ago
      I rarely need to convert markdown documents, but sometimes I need to produce a report for a client or a non-technical internal reader.
      • DidYaWipe 2y ago
        Exactly. The problem isn't converting; it's viewing.
• kreelman 2y ago
  Hmmm. I see the use in this... For me, everything swirls in an enjoyable vortex towards org-mode. - Literate Programming, tangle/weave - Export to DocX, PDF, HTML - Org-Roam - Time Management. Several things mentioned above are day to day. I think spectacular things are often made up of collections of useful everyday things.
  • lincpa 2y ago
    [dead]
  • SirHumphrey 2y ago
    Org was designed for being a PKM-like system. Markdown was designed for README.md. It doesn't mean that you cant use either for the other task, but Obsidian for example had to modify markdown significantly, to add support for tags, math, etc. It's just a shame that org format works really well only in emacs.
  • terminalgravity 2y ago
    Org mode is great for plain text info storage. Just difficult to use if you havent joined the church of emacs
    • renewedrebecca 2y ago
      Why? I use it in VSCode and vim all the time.
• xyst 2y ago
  I adopted obsidian recently to replace notion and it’s been a refreshing change. In its basic state (no plugins), it’s just a bunch of markdown files. Very easy to search notes and even have a dedicated folder for diary entries.
• gerdesj 2y ago
  Mediawiki. Let's balance durability against functionality. MW gets you a massively scalable doc store that does not need much room. Most MW instances are MySQL/MariaDB backed and the schema etc is very well described. Keep it plain text for "notes" but a MW will be easily discoverable for quite some time from now.
  • geokon 2y ago
    https://www.mediawiki.org/wiki/Markup_spec https://www.mediawiki.org/wiki/Markup_spec encouraging..
• xenodium 2y ago
  I built Markdown to web, drag and drop solution: https://lmno.lol https://lmno.lol Here's a demo https://www.youtube.com/watch?v=SykbiVweYH8 https://www.youtube.com/watch?v=SykbiVweYH8
• scubbo 2y ago
  > Even self-hosting isn't foolproof - your content can vanish when you forget to pay for hosting I know what they mean - "running applications that you maintain and deploy yourself, on hardware/platforms that you don't" - but this is strange, to my eyes. If it's running on someone else's hardware (whatever it is), then it's not self-*hosted*, surely? It's self-owned, but not self-hosted?
• TheMode 2y ago
  I have personally started to archive pages I find interesting through a browser extension. Its html/css not markdown but good enough for my needs.
• zackham 2y ago
  I've used or built more personal knowledge/task/project management tools than I care to list over the years, and adopted various methods along the way. I've ended up in a place where I know what I need day to day: A place to dump my ideas, plans, reflections, and tasks, along with methods of processing and accessing all this data. It's hard to compete with plain text files, a notebook, and structured daily/weekly rituals that process these notes into actionable tasks, meeting agendas, and project docs. It's not that time consuming, it's super effective, and most importantly, it's infinitely and freely customizable because instead of software, you just have checklists and processes to manually follow. You can execute GTD without touching a computer: https://gettingthingsdone.com/wp-content/uploads/2014/10/Weekly_Review_Checklist.pdf https://gettingthingsdone.com/wp-content/uploads/2014/10/Wee... I can get by just fine with that system, but a handful of months back I started wanting software again. Reminders, task wrangling, workflows around taking meeting notes, taking and processing transcripts of talking through ideas, automated daily and weekly checkins with summaries, project work logs, managing lists of things to talk about with people, the list goes on.... Same reasons I have always reached for software, and the same reasons I wrote my own system a few times over. But this time I had some new thoughts: - I want this to have a chance at being my last system. For that, I must be able to read/edit the data without special software. I settled on committing to building software that interfaces with folders of Markdown files exclusively. I could use Obsidian to cover any gaps and get work done immediately–I don't need my software to do it all right away. - I want to own as much of my recorded activity/thoughts as possible, so I can drop it into new AI models, giving them a ton of context about me and what I'm up to, and avoid getting vendor locked to OpenAI. - I want ubiquitous access to the system, which means it's gotta be easily used from a phone. 7k LOC later and I've got a Telegram bot with a plugin architecture and a pile of plugins that implement everything I've described and more. The plugin arch means there's a defined interface and every new piece of functionality never ends up with more than 1k LOC in a file. My objective was to structure the project specifically so I could avoid the pitfalls of AI generated code as projects get large. Everything isolated with well defined integration points. I chose Telegram because they have a great API, supporting custom keyboards for quick actions, audio input for taking voice memos that my system transcribes, and reaching out to me with reminders/requests on whatever device I'm on. The result is thousands of messages that have translated into a nicely organized Obsidian vault. Couldn't be happier and think there's a chance I'll live with this thing for the foreseeable future–and I can always swap out the interface away from Telegram, build a proper frontend, or drop it altogether and be left with my Markdown files. If anyone is interested I'd be happy to share what I've got. Just my private project that I'm reaping a lot of benefit from. Here's a quick dump of some of my plugin commands to get a flavor of what I'm talking about: https://gist.github.com/zackham/3c2d061e6dd0127958c913329aa05e31 https://gist.github.com/zackham/3c2d061e6dd0127958c913329aa0...
• profsummergig 2y ago
  Windows here. I use VSCode for markdown. Obsidian's been coming up on the radar often. This post finally made me try it out. I like it a lot. But there's one reason I won't be using it as my main driver for markdown files: I can't open files that are not in a vault. I have markdown files everywhere on my drive. And I don't want to make the entire drive a vault (for various reasons). Obsidian configurable as... 1) my default file handler for markdown files 2) capable of opening and saving markdown files in any location on my PC ...would be sweet. (From my research, it can't do these currently.)
• smm11 2y ago
  WTF? text.txt Readable in everything, since forever.
  • denzil 2y ago
    I wish. If you live in any country that uses more than ASCII, then certainly not since forever. I mean, just for my language there were 7 different encodings (according to Wikipedia, possibly more) before Unicode era. When you want to read these it's solvable problem, but still it is extra work to deal with it. Now that we have UTF-8 as de-facto standard, it is much better, but there are still problems. Like when you use Japanese and it gets displayed as Chinese (same characters are different glyphs depending on language).
• eviks 2y ago
  > The format deliberately avoids precise control over display details like font selection4. Following the rule of least power, I consider this limitation a feature. For contrast, consider PDF - a format so powerful that it can run Doom. Just pick a more relevant format for contrast to see that this is no feature! It's not like PDF is the only alternative
• drivingmenuts 2y ago
  Keep it in the format appropriate to the information. If just the text is important, Markdown is probably fine. If the structure is important, keep it in HTML. If the layout is important, PDF. You wouldn't store a Gutenberg bible in Markdown, would you? (Don't answer that - there's always one asshole who would)
• hamsterbase 2y ago
  When it comes to web archiving, I've found that Markdown has some real limitations. Sure, it's great for basic text, but it struggles with things like embedded content and non-standard layouts. Try archiving a Twitter thread or an app-style webpage in Markdown, and you'll see what I mean. It just doesn't capture the full picture. That's why I've come to prefer formats like webarchive, mhtml, or single HTML files for archiving. They're incredibly faithful to the original content - you get almost perfect rendering of the original page, complete with styling and layout. Plus, they can capture stuff behind paywalls or on logged-in pages, which is a huge plus. The real challenge, though, isn't just about saving the content. It's about making that saved content useful. These archive formats are great for preservation, but they can quickly become a mess of unorganized files that are hard to search through or make sense of. I think the key is finding ways to organize and interact with these archives more effectively. Things like full-text search across all your saved pages, the ability to add notes or highlights directly on the archived content, and smart tagging systems could go a long way. And it'd be really powerful if we could integrate these archives with other knowledge management tools we use. It's an interesting problem space, and I think there's a lot of room for innovation in how we approach personal web archiving and knowledge management.
• hamsterbase 2y ago
  When it comes to web archiving, I've found that Markdown has some real limitations. Sure, it's great for basic text, but it struggles with things like embedded content and non-standard layouts. Try archiving a Twitter thread or an app-style webpage in Markdown, and you'll see what I mean. It just doesn't capture the full picture. That's why I've come to prefer formats like webarchive, mhtml, or single HTML files for archiving. They're incredibly faithful to the original content - you get almost perfect rendering of the original page, complete with styling and layout. Plus, they can capture stuff behind paywalls or on logged-in pages, which is a huge plus. The real challenge, though, isn't just about saving the content. It's about making that saved content useful. These archive formats are great for preservation, but they can quickly become a mess of unorganized files that are hard to search through or make sense of. I think the key is finding ways to organize and interact with these archives more effectively. Things like full-text search across all your saved pages, the ability to add notes or highlights directly on the archived content, and smart tagging systems could go a long way. And it'd be really powerful if we could integrate these archives with other knowledge management tools we use. I develop a tool called HamsterBase that seems to address a lot of these issues we've been discussing. t's a local-first app. That means all your data stays on your own device - no need to worry about your personal archives being stored on someone else's servers. There's no sign-up or registration required, which is refreshing in today's cloud-centric world.
  • thangalin 2y ago
    > [Markdown] struggles with things like embedded content and non-standard layouts. I don't share that experience. I typeset all these documents using Markdown with pandoc's div extension, transformed into XHTML, and then passed to ConTeXt: * https://impacts.to/downloads/lowres/impacts.pdf https://impacts.to/downloads/lowres/impacts.pdf * https://dave.autonoma.ca/blog/2020/04/28/typesetting-markdown-part-8/jekyll-hyde.pdf https://dave.autonoma.ca/blog/2020/04/28/typesetting-markdow... * https://pdfhost.io/v/4FeAGGasj_SepiSolar_Highlevel_Software_Design https://pdfhost.io/v/4FeAGGasj_SepiSolar_Highlevel_Software_... From XHTML, the document is transformed into TeX statements, which opens a world of possibilities. In the following video, custom styling is applied to nested contents: https://youtu.be/3QpX70O5S30?t=35 https://youtu.be/3QpX70O5S30?t=35
• abetancort 2y ago
  PDF/A... It was not that difficult. Don't reinvent the wheel, guys.
• Beijinger 2y ago
  Hm. Great. I save everything interesting. I have a data folder with letters a-z in it. Something interesting might be saved in HTML or PDF under data/a/ai/programming Folders have a problem because the same thing could be saved under data/p/programming/ai But it is a start. For everything else, there is recoll. https://www.recoll.org/ https://www.recoll.org/
• g8oz 2y ago
  Markdown is great... But you know what else is great? OPML. We need more tooling around OPML. It's not being used nearly as much as it should be for Personal Knowledge Management.
• yawnxyz 2y ago
  well, I wish I could have saved all my old Flash sound design and game experiments to Markdown, and still be able to play them
• todotask 2y ago
  My thought on custom Astro components is that they provide a flexible format that can be converted into MD, HTML, JSON and other formats.
• kmarc 2y ago
  Indeed, I also realized that bookmarks are worthless on the long run. When choosing a note taking / knowledge management app, the main decision point was if it has a Firefox extension that can capture a web page into markdown and automatically save into my notes. I used to use Joplin, lately switched to Obsidian. Both offer this functionality.
  • KetoManx64 2y ago
    What drove you to switch to Obsidian? I'm considering it myself and have been playing around with Obsidian the past couple of days after about 4 years of Joplin, 2 of which with a self hosted Joplin Server. I'm tired of basic features being missing and extensions breaking because they're no longer maintained, and basic features like linking between notes while writing a note not being built in.
    • kmarc 2y ago
      Joplin worked great when I spent 8h+ daily on my laptop (computer with big screen and physical keyboard). During my long sabbatical, I wanted to take notes on my phone, a LOT. Joplin sucks at that, clumsy, non-user friendly android client. Tried obsidian (first on mobile) and it is superb. I had to install a couple extensons (S3 sync, "Ink" for drawing with a pen), and it just works. It's so good, I sometimes even edit tables on my phone. With Joplin, note taking on my phone was just dumping thoughts in random formats to it and later fixing it on my desktop. Like Joplin, Obsidian also has a Firefox extension to capture a web page I to markdown. So after a couple days of trial, I realized that all the features Joplin has, obsidian has it too, with a much better (and snappier) UX on both my Linux desktop and Android. The only thing I wish for of it was Open Source. But oh well, I'm not dogmatic about that anymore
• briandear 2y ago
  I don’t like Markdown because I don’t want to remember a syntax. Most normal people I know have no idea what Markdown even is. The idea that I can’t see my formatting when I’m writing is annoying. What’s the point? It’s like MD is writing code and to “see” the document, you have to run it. In other words what you see is not what you get — you only see what you get when “previewing.”
• Sincere6066 2y ago
  But I hate Markdown.
  • blobcode 2y ago
    Then use any of the plethora of alternative text formatting tools (TeX, for one).
  • DidYaWipe 2y ago
    I hate that it's rampant but nobody calls out the near-total lack of READERS for it. WTF? Imagine writing everything in HTML, but there are no browsers to render it. That's basically where Markdown has been forever.
• geokon 2y ago
  It feels like a long term solution would be to use a markdown that is both easy to write (not RTF or XHTML), but has a defined grammar in some standard format (ex: EBNF). Most platform/languages will have a parser and so you can whip up a "renderer" or converter trivially at any point. The only markup I'm finding with a grammar is MediaWiki (sort of..) https://www.mediawiki.org/wiki/Markup_spec https://www.mediawiki.org/wiki/Markup_spec Even Djot doesn't seem to have one. Weird..
  • DidYaWipe 2y ago
    "so you can whip up a "renderer" or converter trivially at any point" And yet almost no one has...
    • Y_Y 2y ago
      To be fair, one there's a good one there's much less incentive to write something new. In this case the good converter is Pandoc: https://pandoc.org/ https://pandoc.org/
      • DidYaWipe 2y ago
        Thanks; but that's a converter, not a viewer. The problem is the lack of READER applications to simply view (not edit or convert) all these Markdown documents.
        • Y_Y 2y ago
          In my defence, the comment I was replying to mentioned "renderers" and "converters". Furthermore IMHO, any text editor is a Markdown reader. If you want it formatted as is it were "markup" then might I suggest converting to e.g. PDF using Pandoc and then using one of the many capable viewers.
          • DidYaWipe 2y ago
            Noted, re the other comment. But... come on. You might just as well say any text editor is a browser, because you can technically read HTML with it. You can also technically read Word documents with a text reader.
• galkk 2y ago
  Markdown is great, but not a panacea. Tables, in particular, just suck, especially if you want to have even slight formatting inside of the cells. Unfortunately, it’s either plain-text-readable or rich representation. Pick your poison.
  • DidYaWipe 2y ago
    The biggest problem with Markdown is the baffling lack of plain VIEWERS. Not editors with a preview pane, but straight-up viewers that render Markdown for reading. There are very, very few. I use Marked 2, for Mac. I don't even remember if I ever found another one. It's irritating as hell, because pretty much every open-source project's read-me files are in Markdown. Why, when there is no viewer anywhere near as ubiquitous as those for PDF... despite Markdown being much simpler and better understood?
    • oneeyedpigeon 2y ago
      Agreed. I use QLMarkdown [0] for preview in Finder and this markdown-viewer extension [1] for in-browser preview. But a standalone, native app would be pretty nice too. [0] https://github.com/sbarex/QLMarkdown https://github.com/sbarex/QLMarkdown [1] https://github.com/simov/markdown-viewer https://github.com/simov/markdown-viewer
      • DidYaWipe 2y ago
        Thanks. Try Marked; it is exactly what I wanted. It's $14, but I decided to reward whoever did what apparently no one else (including me) can be bothered to do. I have tried MarkText, which is yet another editor with a viewer but it's free.
    • pkroll 2y ago
      I use Markdown Viewer, in Chrome: I'd bet there are multiple equivalents in Firefox and Safari. Well. I don't know what Safari's extension universe is like but it seems likely.
    • dredmorbius 2y ago
• whatever1 2y ago
  Unless it’s math in which case you are screwed. Can we have a damn math keyboard and proper character encoding instead of doing shenanigans with latex / office equation editor ? Why in this exact text box I cannot type a differential equation ?
• oneeyedpigeon 2y ago
  Markdown is a wonderful format (I use it all the time) but it's very narrow and I don't think it's appropriate for storing general 'things we might publish'. You lose a lot of semantics just replacing html with markdown. For a general purpose markup language, I don't think we can beat XML.
  • Terr_ 2y ago
    I agree, if the purpose is archival (versus manually reading it with your eyeballs) then you will want a format that (1) can capture information in a somewhat self-documenting way and (2) is in a form that can be easily parsed and converted into a newer format.
• Cilvic 2y ago
  @OP super inspiring. I'm working on a universal capture SDK, a bit like rewind.ai that would make it easy to grab information from screen and then store as Markdown etc. Have you ever wished for something like that?
• tqwhite 2y ago
  Also, as an old person, I will tell you that 1) I got my first personal computer in 1979 and have been trying to keep my bon mots archived ever since. I have tried a million things and have learned one key lesson: It's not really worth it. I literally have a footlocker filled with old disk drives (remember, since 1979!) and I have never, ever gone back more than a few years, hell, more than a year. Now that disks are big, I keep a lot of old stuff. I have, eg, screenshots dating back to 2015. Email before then. And so so much more. I have never gone back more than a few years. I will continue to archive because I must but, Old Person to Young People... Don't put too much effort into long term availability. It's not a good investment.
  • ekanes 2y ago
    Similar perspective, but I'd offer a minor tweak. Just as before gmail people spent a lot of time "managing" their email. Gmail allowed us to stop bothering and just use search to find stuff among the now-messy volume of email. It works pretty well. Similarly, I'd say save everything, but spend no time on organizing it, relying on search and ai/future technology to find what you want from among the mess.
    • infecto 2y ago
      Similar thought after many years of trying to be "organized". Search is what matters, make sure the tool or format of storage allows for easy searching.
    • tqwhite 2y ago
      I'm a big fan of general search.
  • datadrivenangel 2y ago
    A triage system is essential to determining your archival strategy! We can produce information faster than we can produce information storage systems, so we need to be discerning! Random gibberish is less valuable than a screenshot you took in 2015, and that screenshot from a decade ago is probably less valuable than your tax returns or your treatise on the meaning of life! If it's worth keeping, it's worth putting some time into every few years to make sure it's copied somewhere.
• jbd0 2y ago
  The Markdownload browser extension is super useful for saving webpages as Markdown: https://addons.mozilla.org/en-US/firefox/addon/markdownload/ https://addons.mozilla.org/en-US/firefox/addon/markdownload/
  • emptysongglass 2y ago
    I wanted to like MarkDownload, but unfortunately it doesn't auto-add provenance when copy-pasting selected text blocks, at least out of the box.
• mirawelner 2y ago
  Beware for if you continue down this road you will end up sitting in class taking notes in markdown… yes I did do this… I am afraid I am beyond salvation