14 ms·

Managing my personal knowledge base

348 points by tkainrad 7y ago

macawfish 7y ago
The bookmark workflow is an impressive demonstration that browsers have a lot of room for improvement in their bookmarking tools. I'm hoping (and optimistic) that the distributed web brings with it some epiphanies about how to do better local knowledge management!
- jhoechtl 7y ago
  Firefox sync with tags and knowing how to use the FF search bar to filter tags comes close for me
  - dgsb 7y ago
    I didn't know we could search in the bookmarks from the address bar directly. It is indeed a nice feature. Here is the documentation for reference https://support.mozilla.org/en-US/kb/address-bar-autocomplete-firefox https://support.mozilla.org/en-US/kb/address-bar-autocomplet... More specifically in the section Changing results on the fly
nickthemagicman 7y ago
What I would love is a full personal CMS type system, where I can have a subject like "cybersecurity" or "cooking" and I can add bookmarks, notes, pdfs, and they're all treated equally and as first class objects in the system. I would love a system that treats all media the same as just a source of knowledge instead of being hung up on the source type i.e. is it a video, or a book mark or a pdf....
- tkainrad 7y ago
  If you didn't try Notion before, you should probably not judge it solely by my post. It is very versatile and you could probably use it in a way that comes close to what you describe. There is not much support for PDFs or videos though.
- karlicoss 7y ago
  Not exactly CMS, but I'm working on a browser extension along a similar line of thought. It unifies all bookmarks/notes/chat messages etc you have for a certail URL treating them as first-class: https://github.com/karlicoss/promnesia#demo https://github.com/karlicoss/promnesia#demo P.S. It's also relevant to bookmark management as well: you get to see not only notes on the bookmark, but also on the 'child' URLs, e.g. if you open someone's twitter profile you'd be able to see their tweets you've favorited. Or if you open some blog, you'd see posts from that blog you saved on Reddit.
- 1kGarand 7y ago
  I have been using a text file (actually an org mode file) since 2009. Before that I used a plain text file. It doesn't handle video/bookmark/pdf. If I have to save those, I put them in a separate directory and make a note of it in the text file. It's pretty well organized, thanks to org mode. It has a section that is by date, like journal entries, and a section that's by topic. Since it's just a single file, it's very easily searchable. It will never "go down". I don't need to run a database (I tried using wiki software for the same purpose). I can even "link" different sections, by having plain text labels. For example, I can refer to "ZFS Setup 2009-01-01" which is another place in the document I can search for. I can "jump" to sections by using the orgmode annotation. Searching for "* Computers" will go to that top level section. I liked this system so much that I also use it at work. People are very impressed that I can find things in my "notebook" so quickly. It's just text search. Websites disappear. Web apps disappear. Apps disappear. My text file, does not.
  - inakarmacoma 7y ago
    But really working on the go with Android or iOS is painful, no?
    - brushfoot 7y ago
      As someone with a similar setup, I have a Google Doc named "Weekly Scratchpad" with a shortcut on my Android home screen. At the end of the week, I copy anything worth saving into my plain text file(s) and delete the rest.
  - mxuribe 7y ago
    > It doesn't handle video/bookmark/pdf. If I have to save those, I put them in a separate directory and make a note of it in the text file... I've used a similar system in the past (though not based on org mode), but was too brittle if/when "links" broke or were altered. For an over simplistic example, if the references that you made to a video within your master text file(s) breaks because the video's containing folder name has changed even slightly, well that becomes annoying. For a small number of references, types of destination media types, etc., maybe not an issue, but after some point, it can become too much maintenance/annoyance. Is this an issue that you encounter? If so, do you have mitigations for avoiding/resolving the broken links? Just curious.
- inakarmacoma 7y ago
  DynaList?
karlicoss 7y ago
Great writeup! I've got quite similar system in spirit and aims, just instead of lower Notion layer, using org-mode. It ends up searchable, synchronised with all devices, available offline and with great tooling for organizing and processing information. > While documenting my configuration, especially my command-line workflows, I identified some shortcomings Can't agree more! I've considerably simplified so many things while writing up on my setups and publishing code -- I guess it's easier to sneak in unnecessary complexity when it's all in your head. > They are, however, not well suited for keeping extensive bookmarks libraries. Agree state of browser bookmarking is a bit of a shame, interfaces are restricted and bloated with non-functional features. At first I switched to Pinboard [0], but after a while I realized even that was not enough for me and wrote Grasp [1], browser extension to capture stuff directly in org-mode. > Examples of things that should not be Chrome bookmarks Yep, eventually reached exactly the same conclusion! I often want to add private notes, more context etc and it's just not compatible with standard bookmark solutions. > Taking Notes Also big fan of notetaking, I basically write down any remotely meaningful thought if I don't have time to exercise it at the moment (via org-capture). Also using same file for everything and just processing it now and then. Same for ideas for new projects, blog updates, etc -- these are just entries in org-mode. > ..it requires some discipline. I tag and annotate new entries, that I added via the web clipper, about twice a month Yep, doing same, going through clipped links in org-mode, tagging and putting priorities. Then I can sort by priority and start reading/refiling etc. Eventually non-important stuff just sinks down as I don't have time to process everything, but at least it's searchable. > For example, it is quite easy to search through questions you have answered on Stack Overflow. It is also not a problem to go back to your Hacker News posts or search through projects you starred on GitHub. I actually find that even though in theory it's easy, in practice sometimes you don't remember where exactly you need to search for something, so you end up going through 'check reddit saves', 'check HN saves', 'check twitter faves' cycles, etc. So I'm automatically converting these into org-mode and they are searchable as any other org-mode file. I describe it in more detail here [2]. P.S. Great and clean design, especially the sticky navigation on the right! I might borrow the idea :) By the way, I tried in responsive mode and it seems to disappear alltogether, perhaps you could display it on top if the screen is too small? 0. https://pinboard.in https://pinboard.in 1. https://beepb00p.xyz/grasp.html https://beepb00p.xyz/grasp.html 2. https://beepb00p.xyz/pkm-search.html#other https://beepb00p.xyz/pkm-search.html#other
parisidau 7y ago
Any macOS users should check out DevonThink.
- walterbell 7y ago
  Also on iOS.
- Pamar 7y ago
  I have been perpetually sitting on the fence about Devon. My use case: before Google+ folded I saved all my history there, because 99% of this was links to pages on stuff I found interesting/fun/relevant. So now I have a large amount of urls (some will probably be defunct by now but it is not relevant). What I would like to do is to feed these urls to something (DevonThink?) that could access the article, and index it so that if - as an example - I want to prepare a RPG campaign on modern day pirates I can just write in the search box ["pirates" "shipping" "modern"] and hopefully get a list of web articles that are relevant. Now, I know that Devon has some sort of automatic indexing/clustering facility for documents you have on your HD, but it is not really clear to me if it works also with stuff you only have an URL to. (If anyone has some alternatives to suggest I will be very interested - I toyed with the idea of putting together an ElasticSearch VM for this but it remained on the backburner for years). Here is my original request here, btw: https://news.ycombinator.com/item?id=18882167 https://news.ycombinator.com/item?id=18882167
  - Terretta 7y ago
    Maybe https://www.stclairsoft.com/HistoryHound/index.html https://www.stclairsoft.com/HistoryHound/index.html or https://historio.us https://historio.us
    - Pamar 7y ago
      Thanks a lot, I will check these which are totally new to me.
- npx13 7y ago
  +1 for DevonThink, both on mac and ios.
romwell 7y ago
For managing a personal knowledge base, I highly recommend Tiddly Wiki[1] It's a self-contained Wiki/Notebook/Journal with tags, it works instantly, and all of it is in a single HTML file with magic JS in it. Or you can use it with a server. [1]https://tiddlywiki.com/ https://tiddlywiki.com/
- mettamage 7y ago
  I use tiddlywiki with noteself on a self-hosted server.
  - vinu76jsr 7y ago
    how do you secure it, is it a long obfuscated URL?
    - BLKNSLVR 7y ago
      My setup requires a username and password to be entered, which I think is the intended authentication method, since it uses a database at the backend.
  - BLKNSLVR 7y ago
    I've been doing the same for a couple of months now, and I love it. Can access it from various devices through the browser and it syncs well. Big thumbs up from me so far. The hardest part was setting up a CouchDB instance for the backend, but I found some instructions and fiddled with it until it worked. Link: https://noteself.github.io/ https://noteself.github.io/
- tasogare 7y ago
  I strongly support this recommendation. Tiddly Wiki has enough features and extensibility that I wasn’t tempted to write my own bibliography management system after discovering it. Yes, I know there is dedicated software for that but they didn’t fit how I want to structure my notes.
- whalesalad 7y ago
  How do you recommend running it? The install guide lists a half-dozen different possibilities including PHP, Node, Ruby which all look to be vastly different solutions.
antjanus 7y ago
I'm a huge fan of Org-mode and so far it's been the most persistent way to keep track of my knowledge base. I have a single file called "engineering-notebook" and I store pretty much everything in it: - bookmarks to resources - personal notes - personal tutorials/step-by-steps - cheatsheets It works fantastically for me
- sandinmyjoints 7y ago
  It’s all I need, I have tens of thousands of lines of notes at this point, can search them (text, regexes, tags, whatever) essentially instantly, can export to html or slides or pdf or $format, will never go down or close down or disappear behind a paywall, it’s perfect
  - antjanus 7y ago
    I also love that since it's plain text, I can still access it anywhere I want to without having to use Emacs (despite the limited functionality without it).
  - inakarmacoma 7y ago
    Accessible mostly everywhere, but really exploring, manipulating content across mobile devices and operating systems gets pretty frustrating. Love it in theory, but DynaList is just so simple to use anywhere without a second thought.
    - benhurmarcel 7y ago
      Dynalist hits the right spot for me to manage my current work (tasks, quick notes…). Simple but flexible, extra features not overwhelming you if you don't need them. But their mobile app isn't great.
- bachmeier 7y ago
  Why do you use one file? The ability to insert hyperlinks is one of the nice features of org-mode.
chrisweekly 7y ago
See also Roam (https://roamresearch.com https://roamresearch.com)
- k_sze 7y ago
  Interesting. Does anybody know more about the project and the organisation? How do they keep their lights on?
  - mbrock 7y ago
    They’re going to a monthly subscription model when they get out of beta. https://twitter.com/conaw/status/1214855473876201472?s=21 https://twitter.com/conaw/status/1214855473876201472?s=21 “Still working out specifics Right now looking like it'll be $30/month - cheaper for annual - cheaper for students/non-profits/unemployed I personally think unemployed autodidacts should always get student benefits - scam that they don't”
- koevet 7y ago
  Just tried to sign-up on Roam. The sign-up page redirects to some tracking site (https://trackcmp.net/redir?actid=66602... https://trackcmp.net/redir?actid=66602...), which is blocked by my pi-hole. Not looking good.
  - chrisweekly 7y ago
    Bummer. I signed up a few weeks ago (free), and have been enjoying it tremendously. To the point where I'm glad they'll take my money bc it's more than worth it, and I want to feel confident I can rely on it as a customer.
    - swah 7y ago
      It's going to cost around 30 usd/mo so it's something for people that depend on it.
l00sed 7y ago
I love writing an blogging. Started reading into org-mode recently. Wish I had a cleaner setup with my own website. I am pretty satisfied with fox's bookmarks export though. The json it generates is easy to work with and carries lots of useful meta data. I built a little processor to make my bookmarks easily readable and organized here: https://l-o-o-s-e-d.net/bookmarks https://l-o-o-s-e-d.net/bookmarks
abacate 7y ago
I've been using Zim [1] as a knowledge database for around 2 years. I don't like depending on online solutions which may suddenly disappear. Together with some plugins for managing tasks, git and some script for synchronizing repositories, it has been working great. There are some limitations on this approach (ie, two persons editing the same page concurrently is a no-no), but for my use-case it works perfectly. I find it much more intuitive than org-mode, and the fact that it auto-generates a "global" task list based on items spread across the whole notebook makes it much easier to prioritize things. [1] https://zim-wiki.org/ https://zim-wiki.org/
- themodelplumber 7y ago
  I like Zim. For my Batcave I switched from Tiddlywiki to Tomboy to Zim to Notecase Pro and finally switched from that over to files and folders using the Geany editor and a sync service. So far this is the combination that seems to survive software, OS, and environment changes best. Even on Android it's not perfect, but it makes me happier than e.g. trying to use Zim on Android :-)
6nf 7y ago
Off topic sorry but why bother with the cookie agreement on the overlay footer like this? You are not giving users the ability to opt out so this just annoys people without making the website GDPR compliant.
- tkainrad 7y ago
  I will soon disable this for non-EU visitors. A more sophisticated cookie agreement banner is not so trivial to do with static sites. As long as everybody is doing it this way, even big corporations, I will keep this approach for EU visitors.
  - chacha2 7y ago
    It might as well not be there if there's no way to opt out of the cookie. It's not GDPR compliant either way.
svnpenn 7y ago
Where is source code for this blog?
- tkainrad 7y ago
  You can get the general theme here: https://github.com/luizdepra/hugo-coder/ https://github.com/luizdepra/hugo-coder/ My blog itself is not open source. I thought about this for a while, but I think the pressure to keep everything cleaned up, documented, and organized is not worth the benefits in this case. If you have some specific requests, feel free to get in touch and I might be able to help you.
  - svnpenn 7y ago
    I mean yeah, people might want you to do that, but you dont have to. Its your life, and your code. You can post it, and keep it messy if you want to.
Noumenon72 7y ago
The problem with bookmarks is you're never going to have time to reread the whole thing. I make a summary and put the source as a hyperlink so that if I summarized wrong I can go read it again. Summaries are better because you can organize summaries and have everything you know about enums together in a few pages, rather than 20 bookmarks.
- missosoup 7y ago
  You don't need to re-read all of your bookmarks. They're basically a hash table keyed by some concept or some problem that you either list elsewhere or remember in your head, that you can quickly use to pull up the full detail content for the given key. Much like the actual physical bookmarks that are just quick jump references to content you already know the summary of. Don't use bookmarks as a 'I might come back and read this later'. Use them as 'I will need this later'.
  - tkainrad 7y ago
    I fully agree with this sentiment. It is very important to not have the feeling that you need to revisit all your bookmarks. In that case, you will eventually feel overwhelmed and a bookmarks library will do more harm than good.
madhadron 7y ago
I'm surprised no one has mentioned SuperMemo and incremental reading as a means of managing a knowledge base.
- steve1977 7y ago
  Ahh incremental reading in SuperMemo. It makes my Wikipedia addication so much more manageable.
bnj 7y ago
I’d like to look into the features of notion more, but I can’t help but wonder why it isn’t more common to see people using a local SQLite database, or any other self hosted database, to serve as a personal knowledge base management system. A lot of the paid solutions I’ve seen seem to bend over backwards to offer a limited subset of the features that are trivially available in an actual database. Has anyone tried out using a personal database like this?
- tkainrad 7y ago
  During my first weeks of Notion usage, I was repeatedly baffled that I never thought more about personal usage of relational databases, as they make so much sense for organizing notes, and many other things. Having Notion, and also other alternatives, this does not seem viable anymore to me. For example, when I want to edit the "Post" column of my bookmarks database, where I relate blog posts to bookmarks, I just start to type and Notion will suggest matching posts. I can not think of any SQLite frontend that offers such usability features, which is not surprising, as they are clearly not built for using them as personal knowledge base GUIs.
- yen223 7y ago
  I have a postgres instance running somewhere that has a history of stuff that I've recorded. The reason why one might choose not to do this is that a) it takes a bit of work to get syncing across devices, and b) it takes a bit of work to get markdown, images, search, videos etc working.
  - nsomaru 7y ago
    Any chance you could share your process?
- karlicoss 7y ago
  Having database is easy, the hard bit is having tooling to manipulate it
- simonebrunozzi 7y ago
  It might be an unpopular opinion, but I use Notion and think that it's user interface is quite bad. I doubt they will go in the right direction for what I need. I moved to Notion from Evernote (spammy, terrible, no innovation for years) hoping for a better experience, but I am still unsatisfied. I will soon be looking for another solution.
Xunxi 7y ago
I use Microsoft's Onenote. You can literally do everything in the blog as well as audiovisual captures. All text is searchable. This video covers some of the things you can do with it in an academic setting but relatable. https://m.youtube.com/watch?v=JQD5c8A_D2g https://m.youtube.com/watch?v=JQD5c8A_D2g 0:33 - Math equations 1:45 - Replay text 2:45 - Ink to text 3:20 - Research tools 4:39 - Immersive reader 6:01 - Web clipper browser extension 7:12 - Save emails to OneNote NB: https://www.microsoft.com/en-us/microsoft-365/blog/2015/11/12/onenote-in-november-embed-video-record-audio-insert-files-and-more/ https://www.microsoft.com/en-us/microsoft-365/blog/2015/11/1...
- tkainrad 7y ago
  I am sure that there are things OneNote is even better at than Notion. However, some of the most important concepts covered in my post are databases, relations between those databases, and editing workflows (markdown, slash commands). As far as I can tell, OneNote does not even attempt to do these things.
- ssss11 7y ago
  I’ve tried tons over the years and just keep coming back to OneNote. It’s great, very underrated.
- ssivark 7y ago
  I would love to get a Surface and use OneNote -- especially because it has excellent support for hand-written notes (and I much prefer hand-writing to typing). However, I'm hit in the face with two problems: 1. Uncertainty about data export. I don't want to be stuck depending on a third-party proprietary solution for something so important to me. (That said, Microsoft is typically quite good about supporting products for a long time) 2. I actually find Windows unpleasant to use, and much prefer linux. (Arch linux + i3/KDE happens to be my preferred setup).
  - harha 7y ago
    Re 1. I would really not recommend using OneNote. I recently closed an Office 365 Business account and tried to move my notes on similar topics, e.g. pros/cons and how-tos of using different tools, from a group OneNote page to my own and it was very difficult to do so. The documentation isn't very good (it's unclear which version) and I ended up having to install the local version on my Mac and manually copy/past the pages I wanted because the copy notebook dialog didn't work with notebooks that have many pages. The worst part is not being able to access the raw data and simply copy that from one place to another. I've switched to markdown files on Sublime with the Markdown Editing package for now, because they are easier to read and edit, portable and I can convert my notes to wiki pages, blog posts or documentation very easily. I'm still not 100% happy because of the different flavours and how pandoc converts files differently sometimes (especially lists).
analog31 7y ago
I've mentioned this before, but I'd love to have a note taking app as convenient as EverNote, but that lets me insert code cells a la Jupyter notebook. I can barely think without Jupyter any more. Maybe that's not a good thing, but it's sure handy.
- theonething 7y ago
  What do you use Jupyter for? Feel like I'm missing out. :)
  - missosoup 7y ago
    Not OP but I also use jupyter for almost everything. Jupyter notebooks are great for capturing thought process and showing intermediate state of whatever it is you're doing, which means coming back to them even years later it's a lot easier to recall your own thoughts and not have to reverse engineer a monolithic block of code.
  - analog31 7y ago
    I'm a physicist by day, and work for a company that makes measurement equipment. A lot of my work is quantitative, so I'm frequently "thinking" with numbers, equations, graphs, etc. While I'm not employed as a programmer per se, I use scientific programming quite heavily. Jupyter has helped me take notes when I need to "think" quantitatively, and be able to look at answers quickly, plus maintain a record that I can read later on.
- hohloma 7y ago
  Try Joplin, its a direct replacement for EverNote, can import its database, and has active development to add more similar function. The only one that I'm missing right now is OCR, not sure if its coming, though. I'm using dropbox to sync my phone with laptop clients. https://joplinapp.org/ https://joplinapp.org/
  - yoran 7y ago
    +1 for Joplin. I've been using it for over a year now and I love how I maintain control over my notes because in the end, they're just Markdown stored on Dropbox. And you can change backend at any time.
ngcc_hk 7y ago
What is difference of this and bear on iOS and macOS which I used everyday now. Yes depend up server and paid subscription. But use tag and sync across multiple devices
bumbledraven 7y ago
I use SuperMemo (http://www.supermemo.com http://www.supermemo.com).
dross 7y ago
A little surprised that after 64 comments I've only seen 2 instances of Evernote mentioned. I recently went back to using it and have been making use of the clipper fairly often. https://twitter.com/dalevross/status/1214257684863692806 https://twitter.com/dalevross/status/1214257684863692806 https://www.evernote.com/l/AVVHBrGz_9BPMbh_Oxgp8qpibzVtX0tRnug/ https://www.evernote.com/l/AVVHBrGz_9BPMbh_Oxgp8qpibzVtX0tRn... I also found https://dabble.me/ https://dabble.me/ the other day which seemed like a nice slot in for quick thoughts with the ability to email to your journal. I've heard that journaling manually works better but I really dislike writing unless necessary. One of my biggest gripes is with how many places I have my data in currently, and as a dev, I should have done something already but that's another story. I'll definitely have a look at Notion and org-mode having read the feedback here along with a few other mentions elsewhere.
- heavenlyblue 7y ago
  Evernote was useful 10 years ago.
  - gjm11 7y ago
    Would you like to elaborate on what changed over the last 10 years to make it no longer useful?
- metachris 7y ago
  I've worked with Evernote a lot, but handling source code snippets is really painful. Got me switching to Notion too.
- Kihashi 7y ago
  I was a paying EverNote user for years, but they stopped evolving while other NoteTaking apps didn't. I forget which feature they removed that was the last straw, but the recipe front-end was hugely important to me.
kerng 7y ago
I think OneNote is the best option. I have been trying alternatives but nothing seems to come close.
fit2rule 7y ago
Don't bookmark. Instead: Print to PDF. I have every interesting article or web reference I've ever read since 1997, sitting in a PDF file alongside tens of thousands of others. It is so convenient to be able to "ls -l | grep subject" and find all the pages related to that subject, and then to mine the data out of the PDF's for further reference. Or, I just open the PDF and gain the knowledge again. This works very well and doesn't require an active Internet connection. Every year I spend a few weeks up in a mountain retreat, going through the collection and getting an idea for the variety of topics I've read about for the year. Its pretty neat to see, also, the changes in my interests over the years - and as well, its pretty wild to go back to the sites after time and realise I've got the only copy of the site - because its gone now. I'd prefer to see more tools for manipulating PDF's become mainstream in the future. I can't recommend highly enough the productivity I've gained by being able to organise things this way. Its like having my own private Internet of 80,000+ pages, tailored to all the things I'm interested in ..
- lcampbell 7y ago
  I've always thought of PDF as an opaque format. How do you search and/or browse your collection? Does the subject show up in grep [e: without being diligent with filenames]?
  - lloydatkinson 7y ago
    I'd be interesting in knowing this too. It sounds like a good idea.
  - mrspuratic 7y ago
    I (*nix user) use a script that basically does: pdftotext -layout -eol unix -nopgbrk $PDF | egrep ... Many PDFs have compressed content streams, plain text utilities only see metadata in that case. Cached, compressed text-only output is usually tiny, and can be zgrep-ed. pdfinfo shows document metadata (title, subject, keywords and more), but it's quite uncommon for these to be useful (Adobe and LᴬTᴇX-sourced PDFs tend to have this data). Both come with xpdf.
travolter 7y ago
Why would you store all of this valuable information in a proprietary tool? What happens when notion shuts down, jacks up prices, makes feature changes you don't like, etc etc? Make some attempt to migrate all this data? It's gonna be hard to do this in an automated way, if they even have an (export) api for this. I'm using (vim)wiki/gollum and I feel much safer about my data.
- henriks 7y ago
  Fully agree. I was turned off notion for the same reason and reverted back to self-hosting dokuwiki. A much more limited feature set, to be sure, but at least I'll still have my notes 10 years from now.
  - bhl 7y ago
    If there was an open source notion, where would you prefer storage to take place? BYOFS?
    - qot 7y ago
      WebDAV sync would be great. There are plenty of options like NextCloud, which is self-hostable. Sync it to plain text files.
- tkainrad 7y ago
  Obviously, you have a point. I am indeed a little bit worried. Some things that ease my mind: Notion's export and backup features are quite good. You can export everything in various formats, markdown being the most relevant for me. This only becomes messy with complex databases that use formulas and are related to each other. You would not have these with local solutions in the first place. Notion is already profitable and growing rapidly, I do not see them shutting down in the foreseeable future. However, the other concerns are relevant. The API is supposed to be released soon. I intend to either build a backup workflow myself or use other tools that will get developed then.
  - 7y ago
tenant 7y ago
I have a single God ReadMe file in which I try to record all the facts that I should know and 'recipes' that I might need to follow again. Such as that shutting down windows does not dispose of kernel state whereas restart does. Or how to do a certain task on the income tax site or how to set up an odbc datasource in Windows. I've been keeping this file for years and I periodically read through it as much to remind myself of what's in it as anything else. It's saved me a lot of frustration when finding myself having to do things again that I know it took me ages to figure out the previous time but I have since forgotten.
brushfoot 7y ago
I like the Plain Text Project's philosophy. https://plaintextproject.online/ https://plaintextproject.online/ A tool is great for collaboration, but how much do you really need for a personal KB? And how much effort do you want to spend migrating? Then there's vendor lock-in, learning curves, etc. A well-organized directory structure with plain text files gives you a data store that won't become obsolete and can be used on any OS. I use CLI tools to search (e.g., grep) and that's all the features I need right there.
nickjj 7y ago
Does anyone else use plain text notes for everything? I've tried to optimize things so that while working, I can get things jotted down with close to zero resistance. If the barrier of entry is too high then I don't bother doing it. That means often being able to spawn a new terminal, write a command and be done with it. Or to be able to pipe something from my clipboard into an auto-dated file. I ended up putting together this ~15 line Bash script which seems to do the trick: https://github.com/nickjj/notes https://github.com/nickjj/notes I've been using plain text notes since 2001, although I only recently created this script. So far so good.
- giancarlostoro 7y ago
  This is how I use Sublime Text cause I dont have to actually save new files it saves them for me. It loads instantly so its perfect. I paste in code snippets and what have you. I dont use ST3 for proper dev as much cause I use JetBrains IDEs more. I still would prefer something better for my more permanent notes.
- thomas232233 7y ago
  I use emacs org mode which stores every thing in a text file.The text file is fully human readable even without emacs. Its has few rules that makes the collapsing the parts of the text file. One of the main rule is any line that starts with asterik followed by space ("* ") is interpreted as heading and the whole heading can be collapsed.
- mrspuratic 7y ago
  Yep, zero resistance: plaintext, with context, metadata, keywords, minimal formatting, grep and a few scripts. The most important thing to remember is that I don't remember everything, so write it down, and use sensible filenames/hierarchy/tags/metadata so it can be found easily. The less temptation to play with CSS, formatting and markup the better. One day it'll all go in a self-contained minimal wiki. Maybe. On my android phone I use an ancient app called "unote" (installed from an .apk which I keep, it's long since disappeared from Play). Simple, hierarchical plaintext notes, kept in a local sqlite file. Sadly no search, but that just means I have to be disciplined about what, where and pruning.
- 7y ago
dope9967 7y ago
Google Keep is pretty awesome for notes. Has a good search, supports tagging, can pin, manually order notes, create calendar reminders from them, attach images, colors. Supports plain text notes and checklists. Also, there's not only a web app but desktop and phone ones too.
- loriverkutya 7y ago
  And it’s all gone, when they shut it down.
  - dope9967 7y ago
    True. There is an export option for notes in Google Keep, and I don't think they will just shut the service down instantly without warning. I guess there's no silver bullets, every option sucks in a way. You go oldschool with a text document or directory structure, you loose out on usability, if you use any of the services there's a risk of them being shutdown, or you host some service yourself and take on costs of doing so.
w_t_payne 7y ago
I've got a hybrid system. Academic papers, standards documents, white papers etc... get downloaded as PDF and renamed according to a (human-readable) standard scheme that I use (year, title, [author(s)], [paper-type]), and stored in a 'meaningful' filesystem hierarchy. Podcasts and some videos are also stored in the same hierarchy. Documentation for the software libraries and APIs that I use is downloaded from readthedocs (where possible) and stored in a parallel system that takes account of versioning. (So I can concurrently store differently versioned copies of the documentation for a single library). I have a simple python script that iterates through my directory hierarchy and produces a sqlite database and a couple of xlsx files with a row-per document (one spreadsheet for reporting document-management metadata and another that allows me to assign labels and write precis notes). The script also extracts the content of the PDFs as plain-text and feeds some NLP tools that I'm playing with. I use the spreadsheets to keep notes on the files, and the act of manually renaming and sorting the PDFs into the 'right' place in the hierarchy helps me to understand what's in them and remember what I've got. (I'm constantly reorganising the hierarchy as my understanding develops and evolves. My python script keeps everything -- notes and documents and other metadata - in sync). So far this has scaled OK to around 26,000 PDFs.
- HSO 7y ago
  Sounds good. How do you handle cases where the same document fits multiple categories (ie belongs in multiple locations of your hierarchy at once)? Ps. Another question: can you tell which nlp libs are best for this purpose in your experience (and how do you eventually search the generated index?)
  - w_t_payne 7y ago
    I can have multiple copies of the same document in the system. My notes are associated with documents via md5 hash, so it'll link them with all copies that are present. At some point I'll get the script to automatically hardlink or symlink duplicates. To be honest though, trying to decide which location best fits a document is quite a good way to engage with the document content -- so even though decisions are often suboptimal, it isn't really the final hierarchy which is the product here, it's what goes on in my brain during the process of trying to organize the papers. As far as the NLP libs question is concerned -- so far this is just an excuse for me to play with SpaCy .... I'm still at quite an early stage and haven't made much progress (my background is more machine vision than NLP - which I haven't touched since I was an undergrad 20 years ago).
oedmarap 7y ago
For this I use Notable[0] with its data directory located in a GitLab repo. A combination of Markdown and Git with Notable's impressive UI. You can also use any cloud drive, Cryptomator vault, etc. for the storage backend. Notable's categories (tags) for me include things like manpage snippets, code snippets, workspaces/scratchpads, thoughts, etc. Of note, a lot of the things in my head that I need to jot down on my mobile, even at length, are usually stored in Simplenote[1] then transferred to Notable if they're important enough, via Simplenote's desktop app. [0] https://notable.md https://notable.md [1] https://simplenote.com https://simplenote.com
monkin 7y ago
It's too much for me, I couldn't be productive when using few services to store notes. Too much distractions. For my personal needs I use https://github.com/vimwiki/vimwiki https://github.com/vimwiki/vimwiki for notes, ideas, todos, articles etc. I used Buku for few years, but then I realised that I'm not using any bookmarks at all, so moved to native Safari Bookmarks. :) At the end I'm using GIT to have complete archive of each change. :)
crusty511 7y ago
I just use Atom with enhanced-markdown-preview and ascii-tree. It's flexible enough for re-organisation and I can convert to lots of formats with pandoc.
baylessj 7y ago
Glad to see a shoutout to Workona here, I've really appreciated the ability to save bookmarks in workspaces and better separate and compartmentalize my browsing by project. Their support team is great too, had an issue with an update using Brave recently and they were very helpful with fixing the error.
interfixus 7y ago
My note app hopping days ended the moment I first encountered Joplin, which I now use for almost all my writing. Killer feature being the encrypted sync via more or less any kind of storage back end (where presently I use Box via WebDav, but that's easily switched). Seemless work across all my devices, all data in nice and portable markdown. It fits my needs so well that I have taken the extraordinary step of allowing the desktop version on my pc's, even though it is Electron based. https://joplinapp.org/ https://joplinapp.org/
agbell 7y ago
I like plain text backed in google drive. I use The Archive [0] but Notational Velocity [1] is popular as well. I don't have a bookmark system though. This book is an excellent explanation of a plain text knowledge base system [2]. [0] https://zettelkasten.de/the-archive/ https://zettelkasten.de/the-archive/ [1] https://brettterpstra.com/projects/nvalt/ https://brettterpstra.com/projects/nvalt/ [2] https://www.amazon.com/dp/1542866502/ https://www.amazon.com/dp/1542866502/
marvinblum 7y ago
We have build Emvi[1] to solve many of the problems when it comes to managing a long living personal or teams knowledge base. The idea is, just as you describe in your blog post with notions tables, that if you can remember a fraction of the information you're looking for, you can find it using text search, filters and sorting. A fixed structure won't cut it when you have to maintain it for a long time and I believe it's better to add meta data (like tags) to make it searchable. Emvi has a free tier with way higher limits than notion, so that should work for most personal knowledge bases. [1] https://emvi.com https://emvi.com
sbmthakur 7y ago
Has anyone used Pocket for personal knowledge base?
hokkos 7y ago
I also use notion, but I don't put my bookmarks in a database very long, its only for the one I don't have time to categorize, and I try to add them in a neatly organized tree structure.
The_Colonel 7y ago
I was jumping from one note taking app to another before realizing I have to develop my own to be fully satisfied: https://github.com/zadam/trilium https://github.com/zadam/trilium
- soapboxrocket 7y ago
  Looks great, I really like the note map feature.
- thunderbong 7y ago
  This is seriously an amazing piece of software! Thanks for that.
- kamarg 7y ago
  This looks amazing. Any thoughts on what it would take to turn it into a minimally usable PWA so that I could carry things I consider important with me even in offline mode?
  - The_Colonel 7y ago
    No PWA, but there's electron desktop build which can work fully online (and then sync).
- scambier 7y ago
  That looks like an awesome software, and the doc is top notch. I'll try it during the next few weeks.
aizatto 7y ago
For personal/private notes, I built https://www.build.my/logbook https://www.build.my/logbook For public notes, I use GitBook https://www.aizatto.com/why-gitbook https://www.aizatto.com/why-gitbook
kirubakaran 7y ago
You should check out https://histre.com/ https://histre.com/ I've been working on it and imho it is a better solution. With tools like Evernote, Notion etc which are designed to do everything, you end up with a useless write-only "knowledge base" that will make a hoarder blush.
jayaram 7y ago
This has been a workflow that served me well for the last couple of years - Most of my stuff is on dropbox/google drive for accessing them across different devices. I prefer tools that let me control where the data is and not the other way around. For all academic papers, documents, I use Zotero. you can use your favorite pdf reader to annotate or take notes on pdf's and zotero will sync these. I also love the feature where Zotero can automatically extract all annotations from the pdf. (I some times save these as an org file) If I am reading a longform web article or a blog post that I really think is useful and helpful, I also save these to Zotero. The push to kindle extension from fivefilters is an awesome tool that converts webpages to pdf. I'm currently testing the memex extension from worldbrain to annotate and organize my browsing history. For all notes, journals, random thoughts, ideas, (both work and personal), I use orgmode. (I recently switched from ZimWiki). Its been amazing so far. So many things are easier to do orgmode although the learning curve for emacs is pretty steep in the beginning. On mobile, I use the orgzly app for accessing and taking notes! Its by far the best android app I use so far. practices >> tools!
rnikander 7y ago
I use folders full of Omni Outliner documents. I like it, but to me it's missing some features.
JoachimS 7y ago
I'm a bit surprised nobody seem to have mentioned Bear Notes. I've used Onenote, Org Mode in Emacs but I am very happy to have discovered Bear Notes (https://bear.app/ https://bear.app/) last. year. It has really changed how much I document, work with text, store info etc. Bear handles images, lists etc inline. It allows for very easy tagging (creating hierarchies) etc. And it looks really nice. Even with good fonts, Org Mode never really looked pleasing to me [0]. And I found that easthetics is really important for me to actually use the tool. Bear uses SQLite for storage, which can be accessed with any SQLite tool (I've tested it). And it can also do batch export to a number or formats. The exported files looks good and can easily be imported into other tools, for example Org Mode (I've tested it). Bear can also export to PDF with styles. I actually use this to export my notes as reports to the company board, customers etc. And I've gotten compliments on how good the reports looks. ;-) Finally Bear syncs easily. I can now work with my notes on the phone, iPad and laptop. If I only could get the tool to use a solid, non-blinking caret life would be a bliss. Go Bear!
- benhurmarcel 7y ago
  I'm looking forward to try Bear once they have a web client. I need access from Windows/Linux.
thiagomgd 7y ago
Anyone used WikiJS?
rasta78 7y ago
A Todoist app, mac notes or OneNote and pocket to bookmark pages. Why over complicate things!? And if you want to have a framework to memorize things try PolarBookself and Anki flashcards.
jlroger 7y ago
Thanks for the post. Will check out Notion for sure. Although I'm a little worried about the vendor lock-in - as others have mentioned as well. Although I think it is ok to surrender some robustness for the ease of use.

348 points by tkainrad 7y ago

The bookmark workflow is an impressive demonstration that browsers have a lot of room for improvement in their bookmarking tools. I'm hoping (and optimistic) that the distributed web brings with it some epiphanies about how to do better local knowledge management!

jhoechtl 7y ago
Firefox sync with tags and knowing how to use the FF search bar to filter tags comes close for me
- dgsb 7y ago
  I didn't know we could search in the bookmarks from the address bar directly. It is indeed a nice feature. Here is the documentation for reference https://support.mozilla.org/en-US/kb/address-bar-autocomplete-firefox https://support.mozilla.org/en-US/kb/address-bar-autocomplet... More specifically in the section Changing results on the fly

nickthemagicman 7y ago

What I would love is a full personal CMS type system, where I can have a subject like "cybersecurity" or "cooking" and I can add bookmarks, notes, pdfs, and they're all treated equally and as first class objects in the system. I would love a system that treats all media the same as just a source of knowledge instead of being hung up on the source type i.e. is it a video, or a book mark or a pdf....

tkainrad 7y ago
If you didn't try Notion before, you should probably not judge it solely by my post. It is very versatile and you could probably use it in a way that comes close to what you describe. There is not much support for PDFs or videos though.
karlicoss 7y ago
Not exactly CMS, but I'm working on a browser extension along a similar line of thought. It unifies all bookmarks/notes/chat messages etc you have for a certail URL treating them as first-class: https://github.com/karlicoss/promnesia#demo https://github.com/karlicoss/promnesia#demo P.S. It's also relevant to bookmark management as well: you get to see not only notes on the bookmark, but also on the 'child' URLs, e.g. if you open someone's twitter profile you'd be able to see their tweets you've favorited. Or if you open some blog, you'd see posts from that blog you saved on Reddit.
1kGarand 7y ago
I have been using a text file (actually an org mode file) since 2009. Before that I used a plain text file. It doesn't handle video/bookmark/pdf. If I have to save those, I put them in a separate directory and make a note of it in the text file. It's pretty well organized, thanks to org mode. It has a section that is by date, like journal entries, and a section that's by topic. Since it's just a single file, it's very easily searchable. It will never "go down". I don't need to run a database (I tried using wiki software for the same purpose). I can even "link" different sections, by having plain text labels. For example, I can refer to "ZFS Setup 2009-01-01" which is another place in the document I can search for. I can "jump" to sections by using the orgmode annotation. Searching for "* Computers" will go to that top level section. I liked this system so much that I also use it at work. People are very impressed that I can find things in my "notebook" so quickly. It's just text search. Websites disappear. Web apps disappear. Apps disappear. My text file, does not.
- inakarmacoma 7y ago
  But really working on the go with Android or iOS is painful, no?
  - brushfoot 7y ago
    As someone with a similar setup, I have a Google Doc named "Weekly Scratchpad" with a shortcut on my Android home screen. At the end of the week, I copy anything worth saving into my plain text file(s) and delete the rest.
- mxuribe 7y ago
  > It doesn't handle video/bookmark/pdf. If I have to save those, I put them in a separate directory and make a note of it in the text file... I've used a similar system in the past (though not based on org mode), but was too brittle if/when "links" broke or were altered. For an over simplistic example, if the references that you made to a video within your master text file(s) breaks because the video's containing folder name has changed even slightly, well that becomes annoying. For a small number of references, types of destination media types, etc., maybe not an issue, but after some point, it can become too much maintenance/annoyance. Is this an issue that you encounter? If so, do you have mitigations for avoiding/resolving the broken links? Just curious.
inakarmacoma 7y ago
DynaList?

karlicoss 7y ago

Great writeup! I've got quite similar system in spirit and aims, just instead of lower Notion layer, using org-mode. It ends up searchable, synchronised with all devices, available offline and with great tooling for organizing and processing information. > While documenting my configuration, especially my command-line workflows, I identified some shortcomings Can't agree more! I've considerably simplified so many things while writing up on my setups and publishing code -- I guess it's easier to sneak in unnecessary complexity when it's all in your head. > They are, however, not well suited for keeping extensive bookmarks libraries. Agree state of browser bookmarking is a bit of a shame, interfaces are restricted and bloated with non-functional features. At first I switched to Pinboard [0], but after a while I realized even that was not enough for me and wrote Grasp [1], browser extension to capture stuff directly in org-mode. > Examples of things that should not be Chrome bookmarks Yep, eventually reached exactly the same conclusion! I often want to add private notes, more context etc and it's just not compatible with standard bookmark solutions. > Taking Notes Also big fan of notetaking, I basically write down any remotely meaningful thought if I don't have time to exercise it at the moment (via org-capture). Also using same file for everything and just processing it now and then. Same for ideas for new projects, blog updates, etc -- these are just entries in org-mode. > ..it requires some discipline. I tag and annotate new entries, that I added via the web clipper, about twice a month Yep, doing same, going through clipped links in org-mode, tagging and putting priorities. Then I can sort by priority and start reading/refiling etc. Eventually non-important stuff just sinks down as I don't have time to process everything, but at least it's searchable. > For example, it is quite easy to search through questions you have answered on Stack Overflow. It is also not a problem to go back to your Hacker News posts or search through projects you starred on GitHub. I actually find that even though in theory it's easy, in practice sometimes you don't remember where exactly you need to search for something, so you end up going through 'check reddit saves', 'check HN saves', 'check twitter faves' cycles, etc. So I'm automatically converting these into org-mode and they are searchable as any other org-mode file. I describe it in more detail here [2]. P.S. Great and clean design, especially the sticky navigation on the right! I might borrow the idea :) By the way, I tried in responsive mode and it seems to disappear alltogether, perhaps you could display it on top if the screen is too small? 0. https://pinboard.in https://pinboard.in 1. https://beepb00p.xyz/grasp.html https://beepb00p.xyz/grasp.html 2. https://beepb00p.xyz/pkm-search.html#other https://beepb00p.xyz/pkm-search.html#other

parisidau 7y ago

Any macOS users should check out DevonThink.

walterbell 7y ago
Also on iOS.
Pamar 7y ago
I have been perpetually sitting on the fence about Devon. My use case: before Google+ folded I saved all my history there, because 99% of this was links to pages on stuff I found interesting/fun/relevant. So now I have a large amount of urls (some will probably be defunct by now but it is not relevant). What I would like to do is to feed these urls to something (DevonThink?) that could access the article, and index it so that if - as an example - I want to prepare a RPG campaign on modern day pirates I can just write in the search box ["pirates" "shipping" "modern"] and hopefully get a list of web articles that are relevant. Now, I know that Devon has some sort of automatic indexing/clustering facility for documents you have on your HD, but it is not really clear to me if it works also with stuff you only have an URL to. (If anyone has some alternatives to suggest I will be very interested - I toyed with the idea of putting together an ElasticSearch VM for this but it remained on the backburner for years). Here is my original request here, btw: https://news.ycombinator.com/item?id=18882167 https://news.ycombinator.com/item?id=18882167
- Terretta 7y ago
  Maybe https://www.stclairsoft.com/HistoryHound/index.html https://www.stclairsoft.com/HistoryHound/index.html or https://historio.us https://historio.us
  - Pamar 7y ago
    Thanks a lot, I will check these which are totally new to me.
npx13 7y ago
+1 for DevonThink, both on mac and ios.

romwell 7y ago

For managing a personal knowledge base, I highly recommend Tiddly Wiki[1] It's a self-contained Wiki/Notebook/Journal with tags, it works instantly, and all of it is in a single HTML file with magic JS in it. Or you can use it with a server. [1]https://tiddlywiki.com/ https://tiddlywiki.com/

mettamage 7y ago
I use tiddlywiki with noteself on a self-hosted server.
- vinu76jsr 7y ago
  how do you secure it, is it a long obfuscated URL?
  - BLKNSLVR 7y ago
    My setup requires a username and password to be entered, which I think is the intended authentication method, since it uses a database at the backend.
- BLKNSLVR 7y ago
  I've been doing the same for a couple of months now, and I love it. Can access it from various devices through the browser and it syncs well. Big thumbs up from me so far. The hardest part was setting up a CouchDB instance for the backend, but I found some instructions and fiddled with it until it worked. Link: https://noteself.github.io/ https://noteself.github.io/
tasogare 7y ago
I strongly support this recommendation. Tiddly Wiki has enough features and extensibility that I wasn’t tempted to write my own bibliography management system after discovering it. Yes, I know there is dedicated software for that but they didn’t fit how I want to structure my notes.
whalesalad 7y ago
How do you recommend running it? The install guide lists a half-dozen different possibilities including PHP, Node, Ruby which all look to be vastly different solutions.

antjanus 7y ago

I'm a huge fan of Org-mode and so far it's been the most persistent way to keep track of my knowledge base. I have a single file called "engineering-notebook" and I store pretty much everything in it: - bookmarks to resources - personal notes - personal tutorials/step-by-steps - cheatsheets It works fantastically for me

sandinmyjoints 7y ago
It’s all I need, I have tens of thousands of lines of notes at this point, can search them (text, regexes, tags, whatever) essentially instantly, can export to html or slides or pdf or $format, will never go down or close down or disappear behind a paywall, it’s perfect
- antjanus 7y ago
  I also love that since it's plain text, I can still access it anywhere I want to without having to use Emacs (despite the limited functionality without it).
- inakarmacoma 7y ago
  Accessible mostly everywhere, but really exploring, manipulating content across mobile devices and operating systems gets pretty frustrating. Love it in theory, but DynaList is just so simple to use anywhere without a second thought.
  - benhurmarcel 7y ago
    Dynalist hits the right spot for me to manage my current work (tasks, quick notes…). Simple but flexible, extra features not overwhelming you if you don't need them. But their mobile app isn't great.
bachmeier 7y ago
Why do you use one file? The ability to insert hyperlinks is one of the nice features of org-mode.

chrisweekly 7y ago

See also Roam (https://roamresearch.com https://roamresearch.com)

k_sze 7y ago
Interesting. Does anybody know more about the project and the organisation? How do they keep their lights on?
- mbrock 7y ago
  They’re going to a monthly subscription model when they get out of beta. https://twitter.com/conaw/status/1214855473876201472?s=21 https://twitter.com/conaw/status/1214855473876201472?s=21 “Still working out specifics Right now looking like it'll be $30/month - cheaper for annual - cheaper for students/non-profits/unemployed I personally think unemployed autodidacts should always get student benefits - scam that they don't”
koevet 7y ago
Just tried to sign-up on Roam. The sign-up page redirects to some tracking site (https://trackcmp.net/redir?actid=66602... https://trackcmp.net/redir?actid=66602...), which is blocked by my pi-hole. Not looking good.
- chrisweekly 7y ago
  Bummer. I signed up a few weeks ago (free), and have been enjoying it tremendously. To the point where I'm glad they'll take my money bc it's more than worth it, and I want to feel confident I can rely on it as a customer.
  - swah 7y ago
    It's going to cost around 30 usd/mo so it's something for people that depend on it.

l00sed 7y ago

I love writing an blogging. Started reading into org-mode recently. Wish I had a cleaner setup with my own website. I am pretty satisfied with fox's bookmarks export though. The json it generates is easy to work with and carries lots of useful meta data. I built a little processor to make my bookmarks easily readable and organized here: https://l-o-o-s-e-d.net/bookmarks https://l-o-o-s-e-d.net/bookmarks

abacate 7y ago

I've been using Zim [1] as a knowledge database for around 2 years. I don't like depending on online solutions which may suddenly disappear. Together with some plugins for managing tasks, git and some script for synchronizing repositories, it has been working great. There are some limitations on this approach (ie, two persons editing the same page concurrently is a no-no), but for my use-case it works perfectly. I find it much more intuitive than org-mode, and the fact that it auto-generates a "global" task list based on items spread across the whole notebook makes it much easier to prioritize things. [1] https://zim-wiki.org/ https://zim-wiki.org/

themodelplumber 7y ago
I like Zim. For my Batcave I switched from Tiddlywiki to Tomboy to Zim to Notecase Pro and finally switched from that over to files and folders using the Geany editor and a sync service. So far this is the combination that seems to survive software, OS, and environment changes best. Even on Android it's not perfect, but it makes me happier than e.g. trying to use Zim on Android :-)

6nf 7y ago

Off topic sorry but why bother with the cookie agreement on the overlay footer like this? You are not giving users the ability to opt out so this just annoys people without making the website GDPR compliant.

tkainrad 7y ago
I will soon disable this for non-EU visitors. A more sophisticated cookie agreement banner is not so trivial to do with static sites. As long as everybody is doing it this way, even big corporations, I will keep this approach for EU visitors.
- chacha2 7y ago
  It might as well not be there if there's no way to opt out of the cookie. It's not GDPR compliant either way.

svnpenn 7y ago

Where is source code for this blog?

tkainrad 7y ago
You can get the general theme here: https://github.com/luizdepra/hugo-coder/ https://github.com/luizdepra/hugo-coder/ My blog itself is not open source. I thought about this for a while, but I think the pressure to keep everything cleaned up, documented, and organized is not worth the benefits in this case. If you have some specific requests, feel free to get in touch and I might be able to help you.
- svnpenn 7y ago
  I mean yeah, people might want you to do that, but you dont have to. Its your life, and your code. You can post it, and keep it messy if you want to.

Noumenon72 7y ago

The problem with bookmarks is you're never going to have time to reread the whole thing. I make a summary and put the source as a hyperlink so that if I summarized wrong I can go read it again. Summaries are better because you can organize summaries and have everything you know about enums together in a few pages, rather than 20 bookmarks.

missosoup 7y ago
You don't need to re-read all of your bookmarks. They're basically a hash table keyed by some concept or some problem that you either list elsewhere or remember in your head, that you can quickly use to pull up the full detail content for the given key. Much like the actual physical bookmarks that are just quick jump references to content you already know the summary of. Don't use bookmarks as a 'I might come back and read this later'. Use them as 'I will need this later'.
- tkainrad 7y ago
  I fully agree with this sentiment. It is very important to not have the feeling that you need to revisit all your bookmarks. In that case, you will eventually feel overwhelmed and a bookmarks library will do more harm than good.

madhadron 7y ago

I'm surprised no one has mentioned SuperMemo and incremental reading as a means of managing a knowledge base.

steve1977 7y ago
Ahh incremental reading in SuperMemo. It makes my Wikipedia addication so much more manageable.

bnj 7y ago

I’d like to look into the features of notion more, but I can’t help but wonder why it isn’t more common to see people using a local SQLite database, or any other self hosted database, to serve as a personal knowledge base management system. A lot of the paid solutions I’ve seen seem to bend over backwards to offer a limited subset of the features that are trivially available in an actual database. Has anyone tried out using a personal database like this?

tkainrad 7y ago
During my first weeks of Notion usage, I was repeatedly baffled that I never thought more about personal usage of relational databases, as they make so much sense for organizing notes, and many other things. Having Notion, and also other alternatives, this does not seem viable anymore to me. For example, when I want to edit the "Post" column of my bookmarks database, where I relate blog posts to bookmarks, I just start to type and Notion will suggest matching posts. I can not think of any SQLite frontend that offers such usability features, which is not surprising, as they are clearly not built for using them as personal knowledge base GUIs.
yen223 7y ago
I have a postgres instance running somewhere that has a history of stuff that I've recorded. The reason why one might choose not to do this is that a) it takes a bit of work to get syncing across devices, and b) it takes a bit of work to get markdown, images, search, videos etc working.
- nsomaru 7y ago
  Any chance you could share your process?
karlicoss 7y ago
Having database is easy, the hard bit is having tooling to manipulate it
simonebrunozzi 7y ago
It might be an unpopular opinion, but I use Notion and think that it's user interface is quite bad. I doubt they will go in the right direction for what I need. I moved to Notion from Evernote (spammy, terrible, no innovation for years) hoping for a better experience, but I am still unsatisfied. I will soon be looking for another solution.

Xunxi 7y ago

I use Microsoft's Onenote. You can literally do everything in the blog as well as audiovisual captures. All text is searchable. This video covers some of the things you can do with it in an academic setting but relatable. https://m.youtube.com/watch?v=JQD5c8A_D2g https://m.youtube.com/watch?v=JQD5c8A_D2g 0:33 - Math equations 1:45 - Replay text 2:45 - Ink to text 3:20 - Research tools 4:39 - Immersive reader 6:01 - Web clipper browser extension 7:12 - Save emails to OneNote NB: https://www.microsoft.com/en-us/microsoft-365/blog/2015/11/12/onenote-in-november-embed-video-record-audio-insert-files-and-more/ https://www.microsoft.com/en-us/microsoft-365/blog/2015/11/1...

tkainrad 7y ago
I am sure that there are things OneNote is even better at than Notion. However, some of the most important concepts covered in my post are databases, relations between those databases, and editing workflows (markdown, slash commands). As far as I can tell, OneNote does not even attempt to do these things.
ssss11 7y ago
I’ve tried tons over the years and just keep coming back to OneNote. It’s great, very underrated.
ssivark 7y ago
I would love to get a Surface and use OneNote -- especially because it has excellent support for hand-written notes (and I much prefer hand-writing to typing). However, I'm hit in the face with two problems: 1. Uncertainty about data export. I don't want to be stuck depending on a third-party proprietary solution for something so important to me. (That said, Microsoft is typically quite good about supporting products for a long time) 2. I actually find Windows unpleasant to use, and much prefer linux. (Arch linux + i3/KDE happens to be my preferred setup).
- harha 7y ago
  Re 1. I would really not recommend using OneNote. I recently closed an Office 365 Business account and tried to move my notes on similar topics, e.g. pros/cons and how-tos of using different tools, from a group OneNote page to my own and it was very difficult to do so. The documentation isn't very good (it's unclear which version) and I ended up having to install the local version on my Mac and manually copy/past the pages I wanted because the copy notebook dialog didn't work with notebooks that have many pages. The worst part is not being able to access the raw data and simply copy that from one place to another. I've switched to markdown files on Sublime with the Markdown Editing package for now, because they are easier to read and edit, portable and I can convert my notes to wiki pages, blog posts or documentation very easily. I'm still not 100% happy because of the different flavours and how pandoc converts files differently sometimes (especially lists).

analog31 7y ago

I've mentioned this before, but I'd love to have a note taking app as convenient as EverNote, but that lets me insert code cells a la Jupyter notebook. I can barely think without Jupyter any more. Maybe that's not a good thing, but it's sure handy.

theonething 7y ago
What do you use Jupyter for? Feel like I'm missing out. :)
- missosoup 7y ago
  Not OP but I also use jupyter for almost everything. Jupyter notebooks are great for capturing thought process and showing intermediate state of whatever it is you're doing, which means coming back to them even years later it's a lot easier to recall your own thoughts and not have to reverse engineer a monolithic block of code.
- analog31 7y ago
  I'm a physicist by day, and work for a company that makes measurement equipment. A lot of my work is quantitative, so I'm frequently "thinking" with numbers, equations, graphs, etc. While I'm not employed as a programmer per se, I use scientific programming quite heavily. Jupyter has helped me take notes when I need to "think" quantitatively, and be able to look at answers quickly, plus maintain a record that I can read later on.
hohloma 7y ago
Try Joplin, its a direct replacement for EverNote, can import its database, and has active development to add more similar function. The only one that I'm missing right now is OCR, not sure if its coming, though. I'm using dropbox to sync my phone with laptop clients. https://joplinapp.org/ https://joplinapp.org/
- yoran 7y ago
  +1 for Joplin. I've been using it for over a year now and I love how I maintain control over my notes because in the end, they're just Markdown stored on Dropbox. And you can change backend at any time.

ngcc_hk 7y ago

What is difference of this and bear on iOS and macOS which I used everyday now. Yes depend up server and paid subscription. But use tag and sync across multiple devices

bumbledraven 7y ago

I use SuperMemo (http://www.supermemo.com http://www.supermemo.com).

dross 7y ago

A little surprised that after 64 comments I've only seen 2 instances of Evernote mentioned. I recently went back to using it and have been making use of the clipper fairly often. https://twitter.com/dalevross/status/1214257684863692806 https://twitter.com/dalevross/status/1214257684863692806 https://www.evernote.com/l/AVVHBrGz_9BPMbh_Oxgp8qpibzVtX0tRnug/ https://www.evernote.com/l/AVVHBrGz_9BPMbh_Oxgp8qpibzVtX0tRn... I also found https://dabble.me/ https://dabble.me/ the other day which seemed like a nice slot in for quick thoughts with the ability to email to your journal. I've heard that journaling manually works better but I really dislike writing unless necessary. One of my biggest gripes is with how many places I have my data in currently, and as a dev, I should have done something already but that's another story. I'll definitely have a look at Notion and org-mode having read the feedback here along with a few other mentions elsewhere.

heavenlyblue 7y ago
Evernote was useful 10 years ago.
- gjm11 7y ago
  Would you like to elaborate on what changed over the last 10 years to make it no longer useful?
metachris 7y ago
I've worked with Evernote a lot, but handling source code snippets is really painful. Got me switching to Notion too.
Kihashi 7y ago
I was a paying EverNote user for years, but they stopped evolving while other NoteTaking apps didn't. I forget which feature they removed that was the last straw, but the recipe front-end was hugely important to me.

kerng 7y ago

I think OneNote is the best option. I have been trying alternatives but nothing seems to come close.

fit2rule 7y ago

Don't bookmark. Instead: Print to PDF. I have every interesting article or web reference I've ever read since 1997, sitting in a PDF file alongside tens of thousands of others. It is so convenient to be able to "ls -l | grep subject" and find all the pages related to that subject, and then to mine the data out of the PDF's for further reference. Or, I just open the PDF and gain the knowledge again. This works very well and doesn't require an active Internet connection. Every year I spend a few weeks up in a mountain retreat, going through the collection and getting an idea for the variety of topics I've read about for the year. Its pretty neat to see, also, the changes in my interests over the years - and as well, its pretty wild to go back to the sites after time and realise I've got the only copy of the site - because its gone now. I'd prefer to see more tools for manipulating PDF's become mainstream in the future. I can't recommend highly enough the productivity I've gained by being able to organise things this way. Its like having my own private Internet of 80,000+ pages, tailored to all the things I'm interested in ..

lcampbell 7y ago
I've always thought of PDF as an opaque format. How do you search and/or browse your collection? Does the subject show up in grep [e: without being diligent with filenames]?
- lloydatkinson 7y ago
  I'd be interesting in knowing this too. It sounds like a good idea.
- mrspuratic 7y ago
  I (*nix user) use a script that basically does: pdftotext -layout -eol unix -nopgbrk $PDF | egrep ... Many PDFs have compressed content streams, plain text utilities only see metadata in that case. Cached, compressed text-only output is usually tiny, and can be zgrep-ed. pdfinfo shows document metadata (title, subject, keywords and more), but it's quite uncommon for these to be useful (Adobe and LᴬTᴇX-sourced PDFs tend to have this data). Both come with xpdf.

travolter 7y ago

Why would you store all of this valuable information in a proprietary tool? What happens when notion shuts down, jacks up prices, makes feature changes you don't like, etc etc? Make some attempt to migrate all this data? It's gonna be hard to do this in an automated way, if they even have an (export) api for this. I'm using (vim)wiki/gollum and I feel much safer about my data.

henriks 7y ago
Fully agree. I was turned off notion for the same reason and reverted back to self-hosting dokuwiki. A much more limited feature set, to be sure, but at least I'll still have my notes 10 years from now.
- bhl 7y ago
  If there was an open source notion, where would you prefer storage to take place? BYOFS?
  - qot 7y ago
    WebDAV sync would be great. There are plenty of options like NextCloud, which is self-hostable. Sync it to plain text files.
tkainrad 7y ago
Obviously, you have a point. I am indeed a little bit worried. Some things that ease my mind: Notion's export and backup features are quite good. You can export everything in various formats, markdown being the most relevant for me. This only becomes messy with complex databases that use formulas and are related to each other. You would not have these with local solutions in the first place. Notion is already profitable and growing rapidly, I do not see them shutting down in the foreseeable future. However, the other concerns are relevant. The API is supposed to be released soon. I intend to either build a backup workflow myself or use other tools that will get developed then.
- 7y ago

tenant 7y ago

I have a single God ReadMe file in which I try to record all the facts that I should know and 'recipes' that I might need to follow again. Such as that shutting down windows does not dispose of kernel state whereas restart does. Or how to do a certain task on the income tax site or how to set up an odbc datasource in Windows. I've been keeping this file for years and I periodically read through it as much to remind myself of what's in it as anything else. It's saved me a lot of frustration when finding myself having to do things again that I know it took me ages to figure out the previous time but I have since forgotten.

brushfoot 7y ago

I like the Plain Text Project's philosophy. https://plaintextproject.online/ https://plaintextproject.online/ A tool is great for collaboration, but how much do you really need for a personal KB? And how much effort do you want to spend migrating? Then there's vendor lock-in, learning curves, etc. A well-organized directory structure with plain text files gives you a data store that won't become obsolete and can be used on any OS. I use CLI tools to search (e.g., grep) and that's all the features I need right there.

nickjj 7y ago

Does anyone else use plain text notes for everything? I've tried to optimize things so that while working, I can get things jotted down with close to zero resistance. If the barrier of entry is too high then I don't bother doing it. That means often being able to spawn a new terminal, write a command and be done with it. Or to be able to pipe something from my clipboard into an auto-dated file. I ended up putting together this ~15 line Bash script which seems to do the trick: https://github.com/nickjj/notes https://github.com/nickjj/notes I've been using plain text notes since 2001, although I only recently created this script. So far so good.

giancarlostoro 7y ago
This is how I use Sublime Text cause I dont have to actually save new files it saves them for me. It loads instantly so its perfect. I paste in code snippets and what have you. I dont use ST3 for proper dev as much cause I use JetBrains IDEs more. I still would prefer something better for my more permanent notes.
thomas232233 7y ago
I use emacs org mode which stores every thing in a text file.The text file is fully human readable even without emacs. Its has few rules that makes the collapsing the parts of the text file. One of the main rule is any line that starts with asterik followed by space ("* ") is interpreted as heading and the whole heading can be collapsed.
mrspuratic 7y ago
Yep, zero resistance: plaintext, with context, metadata, keywords, minimal formatting, grep and a few scripts. The most important thing to remember is that I don't remember everything, so write it down, and use sensible filenames/hierarchy/tags/metadata so it can be found easily. The less temptation to play with CSS, formatting and markup the better. One day it'll all go in a self-contained minimal wiki. Maybe. On my android phone I use an ancient app called "unote" (installed from an .apk which I keep, it's long since disappeared from Play). Simple, hierarchical plaintext notes, kept in a local sqlite file. Sadly no search, but that just means I have to be disciplined about what, where and pruning.
7y ago

dope9967 7y ago

Google Keep is pretty awesome for notes. Has a good search, supports tagging, can pin, manually order notes, create calendar reminders from them, attach images, colors. Supports plain text notes and checklists. Also, there's not only a web app but desktop and phone ones too.

loriverkutya 7y ago
And it’s all gone, when they shut it down.
- dope9967 7y ago
  True. There is an export option for notes in Google Keep, and I don't think they will just shut the service down instantly without warning. I guess there's no silver bullets, every option sucks in a way. You go oldschool with a text document or directory structure, you loose out on usability, if you use any of the services there's a risk of them being shutdown, or you host some service yourself and take on costs of doing so.

w_t_payne 7y ago

I've got a hybrid system. Academic papers, standards documents, white papers etc... get downloaded as PDF and renamed according to a (human-readable) standard scheme that I use (year, title, [author(s)], [paper-type]), and stored in a 'meaningful' filesystem hierarchy. Podcasts and some videos are also stored in the same hierarchy. Documentation for the software libraries and APIs that I use is downloaded from readthedocs (where possible) and stored in a parallel system that takes account of versioning. (So I can concurrently store differently versioned copies of the documentation for a single library). I have a simple python script that iterates through my directory hierarchy and produces a sqlite database and a couple of xlsx files with a row-per document (one spreadsheet for reporting document-management metadata and another that allows me to assign labels and write precis notes). The script also extracts the content of the PDFs as plain-text and feeds some NLP tools that I'm playing with. I use the spreadsheets to keep notes on the files, and the act of manually renaming and sorting the PDFs into the 'right' place in the hierarchy helps me to understand what's in them and remember what I've got. (I'm constantly reorganising the hierarchy as my understanding develops and evolves. My python script keeps everything -- notes and documents and other metadata - in sync). So far this has scaled OK to around 26,000 PDFs.

HSO 7y ago
Sounds good. How do you handle cases where the same document fits multiple categories (ie belongs in multiple locations of your hierarchy at once)? Ps. Another question: can you tell which nlp libs are best for this purpose in your experience (and how do you eventually search the generated index?)
- w_t_payne 7y ago
  I can have multiple copies of the same document in the system. My notes are associated with documents via md5 hash, so it'll link them with all copies that are present. At some point I'll get the script to automatically hardlink or symlink duplicates. To be honest though, trying to decide which location best fits a document is quite a good way to engage with the document content -- so even though decisions are often suboptimal, it isn't really the final hierarchy which is the product here, it's what goes on in my brain during the process of trying to organize the papers. As far as the NLP libs question is concerned -- so far this is just an excuse for me to play with SpaCy .... I'm still at quite an early stage and haven't made much progress (my background is more machine vision than NLP - which I haven't touched since I was an undergrad 20 years ago).

oedmarap 7y ago

For this I use Notable[0] with its data directory located in a GitLab repo. A combination of Markdown and Git with Notable's impressive UI. You can also use any cloud drive, Cryptomator vault, etc. for the storage backend. Notable's categories (tags) for me include things like manpage snippets, code snippets, workspaces/scratchpads, thoughts, etc. Of note, a lot of the things in my head that I need to jot down on my mobile, even at length, are usually stored in Simplenote[1] then transferred to Notable if they're important enough, via Simplenote's desktop app. [0] https://notable.md https://notable.md [1] https://simplenote.com https://simplenote.com

monkin 7y ago

It's too much for me, I couldn't be productive when using few services to store notes. Too much distractions. For my personal needs I use https://github.com/vimwiki/vimwiki https://github.com/vimwiki/vimwiki for notes, ideas, todos, articles etc. I used Buku for few years, but then I realised that I'm not using any bookmarks at all, so moved to native Safari Bookmarks. :) At the end I'm using GIT to have complete archive of each change. :)

crusty511 7y ago

I just use Atom with enhanced-markdown-preview and ascii-tree. It's flexible enough for re-organisation and I can convert to lots of formats with pandoc.

baylessj 7y ago

Glad to see a shoutout to Workona here, I've really appreciated the ability to save bookmarks in workspaces and better separate and compartmentalize my browsing by project. Their support team is great too, had an issue with an update using Brave recently and they were very helpful with fixing the error.

interfixus 7y ago

My note app hopping days ended the moment I first encountered Joplin, which I now use for almost all my writing. Killer feature being the encrypted sync via more or less any kind of storage back end (where presently I use Box via WebDav, but that's easily switched). Seemless work across all my devices, all data in nice and portable markdown. It fits my needs so well that I have taken the extraordinary step of allowing the desktop version on my pc's, even though it is Electron based. https://joplinapp.org/ https://joplinapp.org/

agbell 7y ago

I like plain text backed in google drive. I use The Archive [0] but Notational Velocity [1] is popular as well. I don't have a bookmark system though. This book is an excellent explanation of a plain text knowledge base system [2]. [0] https://zettelkasten.de/the-archive/ https://zettelkasten.de/the-archive/ [1] https://brettterpstra.com/projects/nvalt/ https://brettterpstra.com/projects/nvalt/ [2] https://www.amazon.com/dp/1542866502/ https://www.amazon.com/dp/1542866502/

marvinblum 7y ago

We have build Emvi[1] to solve many of the problems when it comes to managing a long living personal or teams knowledge base. The idea is, just as you describe in your blog post with notions tables, that if you can remember a fraction of the information you're looking for, you can find it using text search, filters and sorting. A fixed structure won't cut it when you have to maintain it for a long time and I believe it's better to add meta data (like tags) to make it searchable. Emvi has a free tier with way higher limits than notion, so that should work for most personal knowledge bases. [1] https://emvi.com https://emvi.com

sbmthakur 7y ago

Has anyone used Pocket for personal knowledge base?

hokkos 7y ago

I also use notion, but I don't put my bookmarks in a database very long, its only for the one I don't have time to categorize, and I try to add them in a neatly organized tree structure.

The_Colonel 7y ago

I was jumping from one note taking app to another before realizing I have to develop my own to be fully satisfied: https://github.com/zadam/trilium https://github.com/zadam/trilium

soapboxrocket 7y ago
Looks great, I really like the note map feature.
thunderbong 7y ago
This is seriously an amazing piece of software! Thanks for that.
kamarg 7y ago
This looks amazing. Any thoughts on what it would take to turn it into a minimally usable PWA so that I could carry things I consider important with me even in offline mode?
- The_Colonel 7y ago
  No PWA, but there's electron desktop build which can work fully online (and then sync).
scambier 7y ago
That looks like an awesome software, and the doc is top notch. I'll try it during the next few weeks.

aizatto 7y ago

For personal/private notes, I built https://www.build.my/logbook https://www.build.my/logbook For public notes, I use GitBook https://www.aizatto.com/why-gitbook https://www.aizatto.com/why-gitbook

kirubakaran 7y ago

You should check out https://histre.com/ https://histre.com/ I've been working on it and imho it is a better solution. With tools like Evernote, Notion etc which are designed to do everything, you end up with a useless write-only "knowledge base" that will make a hoarder blush.

jayaram 7y ago

This has been a workflow that served me well for the last couple of years - Most of my stuff is on dropbox/google drive for accessing them across different devices. I prefer tools that let me control where the data is and not the other way around. For all academic papers, documents, I use Zotero. you can use your favorite pdf reader to annotate or take notes on pdf's and zotero will sync these. I also love the feature where Zotero can automatically extract all annotations from the pdf. (I some times save these as an org file) If I am reading a longform web article or a blog post that I really think is useful and helpful, I also save these to Zotero. The push to kindle extension from fivefilters is an awesome tool that converts webpages to pdf. I'm currently testing the memex extension from worldbrain to annotate and organize my browsing history. For all notes, journals, random thoughts, ideas, (both work and personal), I use orgmode. (I recently switched from ZimWiki). Its been amazing so far. So many things are easier to do orgmode although the learning curve for emacs is pretty steep in the beginning. On mobile, I use the orgzly app for accessing and taking notes! Its by far the best android app I use so far. practices >> tools!

rnikander 7y ago

I use folders full of Omni Outliner documents. I like it, but to me it's missing some features.

JoachimS 7y ago

I'm a bit surprised nobody seem to have mentioned Bear Notes. I've used Onenote, Org Mode in Emacs but I am very happy to have discovered Bear Notes (https://bear.app/ https://bear.app/) last. year. It has really changed how much I document, work with text, store info etc. Bear handles images, lists etc inline. It allows for very easy tagging (creating hierarchies) etc. And it looks really nice. Even with good fonts, Org Mode never really looked pleasing to me [0]. And I found that easthetics is really important for me to actually use the tool. Bear uses SQLite for storage, which can be accessed with any SQLite tool (I've tested it). And it can also do batch export to a number or formats. The exported files looks good and can easily be imported into other tools, for example Org Mode (I've tested it). Bear can also export to PDF with styles. I actually use this to export my notes as reports to the company board, customers etc. And I've gotten compliments on how good the reports looks. ;-) Finally Bear syncs easily. I can now work with my notes on the phone, iPad and laptop. If I only could get the tool to use a solid, non-blinking caret life would be a bliss. Go Bear!

benhurmarcel 7y ago
I'm looking forward to try Bear once they have a web client. I need access from Windows/Linux.

thiagomgd 7y ago

Anyone used WikiJS?

rasta78 7y ago

A Todoist app, mac notes or OneNote and pocket to bookmark pages. Why over complicate things!? And if you want to have a framework to memorize things try PolarBookself and Anki flashcards.

jlroger 7y ago

Thanks for the post. Will check out Notion for sure. Although I'm a little worried about the vendor lock-in - as others have mentioned as well. Although I think it is ok to surrender some robustness for the ease of use.