7 ms·

The Sad State of Personal Knowledgebases

115 points by zzzmarcus 11y ago

ckluis 11y ago
Amen. I currently use txt files & Tree 2 (mac app). I’ve tried Quiver, Quip, & Workflowy and cannot find the perfect app to suit this space for my usage patterns. I’m almost 100% back to text files & sublime text.
- dheera 11y ago
  The problem is really that every time something new and better comes around, it becomes a mess to port everything over, especially if it is conceptually different (e.g. tag-based or something else). Dying products also seldom give you a convenient means to export your content. So ultimately the only thing that can last for decades, be cross-platform, and not be tied to one company's product, is text files and directories. For now, throw it all on Dropbox, Github, or whatever you like. When Dropbox and Github die, the file-directory structure will be easy to port to whatever is next.
  - daviross 11y ago
    This has been my current approach for a while, after exporting everything out of Tiddlywiki when it looked like that was dying for a while (Browser support drying up, etc). Now that it's been revived with Tiddlywiki5 and development's picked back up, I'm somewhat tempted to get back into it. Main advantage: Better crosslinking, rich content, etc. Main disadvantage: Less simple to edit, some tedium to export. I'd love to have a Tiddlywiki-like frontend to a set of Markdown text documents backing it. (This is something that's been gnawing at the back of my mind long enough that I might have to buckle down and make it myself) Maybe Markdown + JSON for rich content data...
    - dheera 11y ago
      Where I need markup, I've been sticking to HTML or LaTeX syntax for the most part. Partly because I hate Markdown due to its inconsistent syntax (that's my personal opinion), but also because it feels like a fad of recent years, much like a successor of the MediaWiki fad and the BBcode fad. I feel like in another 2 or 3 years someone will invent Markleft or Markright or Visual Markdown#.NET 2.0 or Dustindown that's 10 times better than Markdown, and then I'll have to switch everything over yet again. HTML and LaTeX on the other hand, aren't dying any time soon due to their extensive deployment, and I'm confident that my syntax will be renderable for at least another 10 years or more.
      - hammerandtongs 11y ago
        Emacs and orgmode are probably your next step :) orgmode is really a superset of markdown with PIM markup added. It exports to markdown and a ton of other formats. Cheers.
      - ta0967 11y ago
        i'm not sure you'd consider it better markdown (i do), but do you know about reStructuredText? it seems to even predate markdown. http://docutils.sourceforge.net/docs/user/rst/quickstart.html http://docutils.sourceforge.net/docs/user/rst/quickstart.htm...
nwatson 11y ago
I like Evernote for a lot of personal stuff, though that's one of the "worser" systems OP lists. Atlassian's Confluence does very well at being a wiki and though you're hostage perhaps to your original layout / organization, at $10/year for self-hosted it's a very good deal. It can store and index PDF, Word, Excel, etc., documents very well. I don't think it does handwriting recognition. If I were to evolve the product category I'd use Confluence as a basis. In fact I'd want to toy with using Confluence (+ Postgres backend) as the basis for any I.T.-related product -- write plugins for user input, generate all reports to Confluence. I'm not sure what licensing restrictions might be but Confluence provides a lot of useful UI for $10/year (structured + unstructured) for any customer, and building on top of that seems like a no-brainer. Perhaps this is what others do with Drupal and other CMS products, though ... I'm not familiar with that model.
- djur 11y ago
  I'm surprised to read this. Confluence is the most unpleasant "wiki" product (its lack of support for redlinks prevents me from really considering it a wiki) I have ever used. Its built-in hierarchical structure for pages is excessively intrusive and undercooked at the same time, and the search is so bad that I have had it fail to pull up pages when I typed in the literal, complete title. Like most Atlassian products, its only apparent notable quality to me is that it integrates with other Atlassian products, as well as being a semi-functional platform on which to pile plugins and customization. It's possible that there's a good Confluence install out there, probably administered by the same people who run that really nice JIRA instance that people claim to have used once.
  - eponeponepon 11y ago
    Confluence is... okay. I think Atlassian have successfully managed to sand down a lot of the sharp corners on older wiki-type packages that tend to act as a barrier to the non-technical types - from my perspective, that seems to mean that my non-technical colleagues can be persuaded to contribute to documents held in Confluence markedly more easily than to contribute to a MediaWiki instance that we previously had. So, while Confluence itself as a piece of software makes me grumble more often than not, I'm more than happy to trade that off against the easier sell to others.
herge 11y ago
My saving grace at work has been the tidily wiki (http://www.tiddlywiki.com http://www.tiddlywiki.com). It's the easiest place to put one-off shell command line snippets, or instructions or whatever. It's search is fast, and the formatting is nice. The only downside is I have to remember to save and copy the downloaded file to google drive every time I edit anything.
- ahnick 11y ago
  Yeah tiddlywiki is by far my preferred way of storing this type of information. Coupled with a FreeNAS box I can't really think of a better way to store this type information.
  - eponeponepon 11y ago
    I never managed to get tiddlywiki working right - pity, because I really wanted it to work for me, but couldn't seem to get to a stage where I'd trust it to have saved something when it said it had. This was five years ago at least, mind - I've no idea what the problem might've been, and even if it wasn't user error, I suppose it might've evaporated by now. (note to self: try tiddlywiki again soon)
- fsiefken 11y ago
  Have you tried the firefox save plugin? You could use it to save to a synched google drive so you don't have to think about that.
- cmrx64 11y ago
  I love love love tiddlywiki. I've been using it for probably 6 or 7 years now. It's simply amazing.
cdumler 11y ago
I commented on Discus the site, as well. I've been a long time user of DEVONthink. It can be used for many purposes, including document organizer, email, GTD, tagging, etc. My favorite feature has always been the Magic Hat, which will suggest under what folders a document should be filed. I clip most interesting web pages, and it is surprisingly accurate once you have established a few documents. http://www.devontechnologies.com http://www.devontechnologies.com
- look_lookatme 11y ago
  I agree, it's great. I just wish DEVONthink had a better iOS client.
bachmeier 11y ago
In my opinion, this is not a "problem" that can be solved. What makes something good varies a lot from one person to the next. In my case, I'd need a way to hold equations, computer code, PDF versions of papers, handwritten notes that someone else has given me, and a lot of other things. I want it stored in a Git repo. I want to view it in a browser. I want it to be convenient and I want it automated. The "answer" is a feature-filled scripting language that allows you to build a customized solution.
- civilian 11y ago
  What about it would you want automated? What kind of scripts would you add to your personal knowledgebase?
  - bachmeier 11y ago
    It's not so much what I would want automated as what I have already automated. As an example, consider how I handle notes that I leave for myself as I'm working. These are things like paragraphs to be added to a paper, ideas for a research topic, a list of things I have to do around the yard, or anything not directly related to the task at hand. I have written a script in D (my programming language of choice). I have a web interface to all of the notes I've jotted down in the past. I can click a topic and see all notes related to that topic. I click the new note button, and it opens Emacs with the file that will hold the note. I type in a note in markdown, and as soon as I close Emacs, the note is converted to html. If I want to change it, I hit the edit button, Emacs opens up, and I make my edits. Upon closing Emacs, the html is rebuilt. Anytime I create a new note or edit an existing note, there is a commit to the Git repo. That's what I mean by automated. I write scripts that handle everything except the content of the notes. I have written or am writing scripts for just about everything I do.
cypherg 11y ago
Confluence for the simple win
irln 11y ago
I've found that using google docs works well if you frame the document title in the form of a question or statement: - What is a JavaScript constructor? - making rvm permanent in rvspec - postgresql listing, creating and dropping a db Some of the "answers" are just a single line. Then all I have to remember is either something in the title or the gist of the thing I need to know and google docs search provides me with a list of related stuff.
pullo 11y ago
Evernote. Free and accessible across many devices. For me it has been best of the worst. It has been my one place to store all information. Evernote serves well as a general knowledge base, but fails for storing programming snippets and ideas.
oxplot 11y ago
How about Camlistore by Brad Fitzpatrick [1]: "Camlistore is a set of open source formats, protocols, and software for modeling, storing, searching, sharing and synchronizing data in the post-PC era. Data may be files or objects, tweets or 5TB videos, and you can access it via a phone, browser or FUSE filesystem." [1]: http://camlistore.org/ http://camlistore.org/
- pjc50 11y ago
  Camlistore is a nice append-orientated sync system, but I wasn't aware it included any data modelling or search?
- epaulson 11y ago
  Camlistore is where I'm expecting to go. They've got some basic things modeled in their world, and the rest I'll fill in using inspiration from schema.org. My plan is to land PDFs of research papers, web bookmarks (and archives of those pages), ebooks, book scans, textfiles/markdown documents, imports of social networks (favorites from Twitter, mostly) and scans of written notes I make to myself in Camlistore. From there, sync to devices and publish to the web as appropriate. I don't expect that I'll ever find the time to build this all myself, so I'm hopeful that either my data will be in a format that other Camlistore apps can read, or if I get the format wrong I'll be able to write a translator to whatever the Camlistore community decides to model for that specific data type.
gkya 11y ago
Org-mode. A benevolent beast. Once one gets to learn it, it can be made to do anything. Also having a programming language shared with the runtime and other programmes for that runtime (Elisp and Emacs) is just priceless, it just lends itself to be customized in whichever way one would like. A major con for me, though, is the lack of decent iOS-Android integration.
- kristiandupont 11y ago
  I've been wanting to learn org-mode for years but I never get around to it. I use something similar, but I only worry that if I were to move to something else I would want it to be in the cloud so I could access it from devices and other laptops. Do you have a solution for this?
  - eeZi 11y ago
    It's just plain text files, so you can sync it using whatever you want. Many people use Dropbox.
  - spatten 11y ago
    I have a bunch of org-mode files that I save into a Dropbox folder. That gives me backups, and access from other computers / laptops. Search is via ack or grep. It theoretically also gives me access from my phone using MobileOrg[1], but I have never actually set that up. [1]: http://orgmode.org/manual/MobileOrg.html http://orgmode.org/manual/MobileOrg.html
    - gkya 11y ago
      > ack or grep C-c C-j. You're missing out :)
      - spatten 11y ago
        Cool, didn't know about that. Does it search multiple files in a directory? I use it for work logs and keep one file per week.
Alex3917 11y ago
In my opinion, FreeMind is the best solution currently out there because of forces mindmapping conventions and places a strong emphasis on folding. A lot of other solutions I've seen are more like concept maps, which are great for project management, but not great for knowledge bases. http://freemind.sourceforge.net/wiki/index.php/Main_Page http://freemind.sourceforge.net/wiki/index.php/Main_Page (The one thing of note is that there is a bug in the default version of OS X Java, so it frequently crashes unless you update to Java 8 and edit the config to use that version of Java.)
- zzzmarcus 11y ago
  Freemind is great for mindmapping in the context of a specific topic or project, but isn't a general purpose knowledgebase. A mindmap starts at a central node and moves outward. This makes it limited in the practical number of nodes you can have--anything over a couple hundred is hard to manage. It also makes it difficult for one node to appear in multiple places within the mindmap.
- jpfed 11y ago
  >A lot of other solutions I've seen are more like concept maps, which are great for project management, but not great for knowledge bases. I've never understood this preference; would you be willing to explain it? It seems almost exactly backward to me: I model tasks as trees of sub-tasks, and knowledge as a general graph of connected concepts.
  - Alex3917 11y ago
    So all knowledge can be seen as connected, but all knowledge can also be put into hierarchies. Concepts and how those concepts are related is something that people come up with, rather than being an inherent property of the universe. The idea of forcing yourself to put knowledge into hierarchies is that it makes your thinking more clear, makes it easier to remember where things are, and perhaps paradoxically makes it easier to see how things are related. (Or at least highlights certain types of relationships that are only visible in the context of hierarchies.) I'll admit it's pretty weird at first to only get one root node, I think it took me a couple years to get used to that. But once I embraced it the benefits quickly became clear.
space_ghost 11y ago
I've been using Zim (http://zim-wiki.org http://zim-wiki.org) for several years, backed up to Dropbox. It's a significant improvement over plain text files.
pjc50 11y ago
OneNote?
- douche 11y ago
  I've become a big fan. With the OneNote clipper extension for Chrome, it's how I bookmark everything I might want to read later.
msluyter 11y ago
I've been using nvAlt for a while. Very fast search and ease of use. Not sure how well it scales in the long run but thus far I've had no problems. Edit to add: Some other interesting features. You can sync to Simplenote automatically, and notes can be saved as text files for easy import/export. There is some markdown support (though I see a number of bugs in the preview). The main downsides appear to be: limited color schemes, no syntax highlighting for markdown, development appears to have stalled.
zwischenzug 11y ago
I have a git repo with notes on all the things I'm interested in, a few larger files (some books I always reference, some images etc.). Over time I've built up some scripts to help with general tasks, eg list TODOs etc.. It's perfectly moulded to my needs. For example, when I find a new utility I run mk_notes.sh [thing], which sets up three files: - links (for relevant web links) - thing.txt (which has basic info about it) - cheat_sheet.txt (which has quick useful things I want to keep) These files suffice for the majority of cases. I used to have more, but found that was over-engineered. I'm not sure I could buy such a thing, and if I did, I'd get irritated with some aspect of it sooner or later. Search is easy (grep et al), git means it's available everywhere (for me). It's literally changed my life, since I now don't 'lose' any research, and can pick up threads later and build.
- eeZi 11y ago
  That sounds a lot like the org-mode concept.
  - zwischenzug 11y ago
    Interesting; a quick glance at the home page, and it looks like it's for planning. I'm happy using JIRA for that, where I have customized workflows. But again, this is just because I'm familiar with JIRA, not because it's necessarily the best tool. Also, I'm a vi user :)
    - eeZi 11y ago
      That's not mutually exclusive - see https://news.ycombinator.com/item?id=10739693 https://news.ycombinator.com/item?id=10739693 Most people use it for planning, but it's also a very nice personal wiki. The great thing is that you can define your tasks inline, if you want.
PaulHoule 11y ago
I use Microsoft's OneNote for this. The search works amazingly well and is fast.
- howeyc 11y ago
  I've been tempted by this, but I have some reservations: 1. Does it work under less-used OSs? Linux/BSD..? 2. Is it a proprietary format? Will you be able to open the file in a decade? My text files will be able to be opened, I'm not sure anyone can still open their MSWorks files. Maybe I'm crazy to worry about opening these files after a decade since using them probably isn't worth while unless you keep up with it. However, I've got files from the 90's that I've been transitioning from machine-to-machine since storage costs keeps getting cheaper.
  - eeZi 11y ago
    No, and yes. It's proprietary and Windows/Android/iOS-only.
    - falcolas 11y ago
      Toss Mac in there as well: https://itunes.apple.com/us/app/microsoft-onenote/id784801555?mt=12 https://itunes.apple.com/us/app/microsoft-onenote/id78480155...
    - PaulHoule 11y ago
      That's false, you can download documentation for the OneNote file format here https://msdn.microsoft.com/en-us/library/dd924743(v=office.12).aspx https://msdn.microsoft.com/en-us/library/dd924743(v=office.1... it is only about 5% of the size of the Adobe PDF specification and I am sure there are corner cases, but it is not too bad.
      - eeZi 11y ago
        I admittedly did not know that they published a specification . It's still a proprietary file format though. Proprietary does not mean "binary and undocumented", it means that it's not an open standard.
howeyc 11y ago
Any reason a bunch of text files in directories wont work? Any "advanced" text editor that can search directories should work: Notepad++,Vim,grep,... anything really. I've been using a bunch of synced markdown files. Works for me, I just have a Markdown editor and a "grep-like" app for android. I'm not sure what you can do for iPhone though. When I've talked to people they couldn't seem to "browse for files/folders" which I thought was strange. The one thing I'm missing is I'd like to "tag" my notes. I haven't decided how/if I'm going to do that yet.
- EC1 11y ago
  I use this [1] to do a simple full-text search of my personal knowledge base. [1] https://github.com/slashdotdash/jekyll-lunr-js-search https://github.com/slashdotdash/jekyll-lunr-js-search
- zzzmarcus 11y ago
  One of the main reasons for using a personal knowledgebase is surfacing patterns in what you've saved. If there is no way of visualizing the relationship between notes/nodes outside of a flat list, pattern finding is limited to what you can remember to search for. Basically the core problem isn't saving and searching for data, it's having great ways to structure and visualize it.
  - howeyc 11y ago
    Good point, unless I know a search term it doesn't work out great. I've been thinking of switching to some kind of wiki format, like maybe creole. Then finding an app that can understand links between files or something.
newman314 11y ago
I used to use Delicious but that didn't solve the "search the content of my bookmarks" problem for me. I've been using Pocket for now and it seems to work fine.
hackercomplex 11y ago
imho the best one ever made is called getguru.com. It's brilliant actually but the team there has priced themselves out of the market charging $7/head as a b2b app because they're from and enterprise sales background and therefor don't realize guru's monster potential in consumer-web. They're happy taking the low hanging fruit $$ from call-centers. sigh
zeveb 11y ago
I've gotta say that org-mode and a git repo do pretty well by me for this kind of thing.
dalacv 11y ago
I like trunknotes (http://www.trunknotes.com/ http://www.trunknotes.com/)
eponeponepon 11y ago
I'm inclined not to see this as surprising. There are as many ways in which people conceptualise their 'offline' knowledge as there are people, and there is never going to be one size to fit all. Org-mode is probably the closest contender, if only for its sheer flexibility - but the fact that it's wed to something (i.e. emacs) that the gen-pop would immediately recoil from is only ever going to limit its adoption to a tiny subset.
biztos 11y ago
I think vendor lock-in is at least as big a problem as searching and scaling. Arguably bigger for this audience: if I have easy unfettered access to the data itself I can always try something else for search/scale. I've tried a few and at least twice tried to write my own (and gave up due to scope creep). Right now I just have a bunch of Markdown files and a giant messy DropBox folder and an almost-as-messy set of private GitHub repos. And photos. And videos. And e-mail. And backups. (What does one do with a Zip disk these days?) I bet a lot of people my age (mid-40's) or older have the same problem: choosing or building a system we think would work long-term, and then gathering and organizing all the "legacy" stuff (which you definitely want!) is already an enormous undertaking, and so instead we live with a bunch of ad-hoc systems that aren't in any way cross-referenced. Nobody wants to be the sysadmin of their own junk drawer, but I for one don't want years' worth of my stuff locked into somebody's Cloud du Jour. (Then again, I have this same problem with actual paper documents, so maybe it's just me.)
- xbryanx 11y ago
  This. As a near 40 person, I now need to search in 5-10 places for that thing I know I wrote down in Markdown, Vim Notes, Workflowy, Tiddlywiki, 1Password, Confluence, or email at some point in the last 20 years. But maybe we just need to give up on finding one tool to store them all. Instead we need to build more federated search across all these platforms. That way I can have one searcher that returns results all my various old "brains."
- dangoor 11y ago
  I have had very similar thoughts, but I have to wonder if it turned out to be wrong. Jerry's Brain: http://jerrysbrain.com/ http://jerrysbrain.com/ Apparently started in TheBrain in 1997 and now has more than 250K "thoughts". That's breathtaking for both the longevity and the scale. I'm guessing that most mind mapping programs wouldn't handle that well. Gathering the legacy stuff is not as important as getting started with something, I think. The legacy will make it into a new system as it's needed. I've been using Evernote for a number of years. I don't particularly like it (cumbersome UI without great navigation), but the search has worked well enough for me to find things and its ubiquity has been great.
pfista 11y ago
Does anyone have a system for managing their relationships? Kind of like CRM but on a personal level. Keeping track of people's details - like their family, their passions, their history etc.
deleted 11y ago
[deleted]
eeZi 11y ago
I, too, am an org-mode fan, but there are a few good ones missing: - Zim (http://zim-wiki.org/ http://zim-wiki.org/, Open Source) - WikidPad (http://wikidpad.sourceforge.net/ http://wikidpad.sourceforge.net/, Open Source) - ConnectedText (http://www.connectedtext.com/ http://www.connectedtext.com/, Windows-only, $$$)
karmacondon 11y ago
I've been working with own "PK" for about a year now. It's been an interesting experiment so far. My concept is based around random re-exposure to information. For whatever reason, the thought of "Let me look through my old bookmarks" never occurred to me and wasn't part of my daily workflow. So one aspect of my system is that it randomly selects bookmarks from different categories every day and shows them to me. It also has spaced repetition for facts and miscellany, or things I'm trying to learn about (like the US Presidents or Shakespeares plays), vocabulary, etc. There were a few interesting things I've noticed by randomly viewing old bookmarks. The first is that linkrot is a very real thing and the web is decaying at a measurable rate. Second, it's amazing how limited my memory is. Half the time I don't remember something I bookmarked from years or even months ago at all. It's new to me, all over again. I'm going to bookmark this discussion and then completely forget it existed within a few months at most. Regularly reviewing old web pages and notes makes me feel like my mind is a sieve, as well as highlighting how my interest and thought processes have changed over the years. I'm really interested in this topic, feel free to contact me to discuss.
- bovermyer 11y ago
  That's an interesting concept. Is the selection of random bookmarks completely random, or does it use a more complex algorithm to choose what to display?
  - karmacondon 11y ago
    There are simple rules, like "Don't show the same bookmark again within X months", where X becomes progressively longer the more times the bookmark has been shown. I can also flag the best stuff so it shows up more often. I did my best to group the text of the urls, using first kmeans in php (which is a very stupid idea) and finally settling on LDA. I'd like to use hierarchical LDA to automatically discover topics within my bookmarks, but that's "Future Work" for now.
- djhn 11y ago
  Sounds like what I have been doing with Evernote. I described the system in a previous discussion: https://news.ycombinator.com/item?id=10301205 https://news.ycombinator.com/item?id=10301205 What you say about review resonates with me. Several years ago I got excited about Spaced repetition software (like Anki, discussed recently on HN) and tried writing an Evernote app that would periodically show me interesting things. Their API didn't work very well for this, so I stuck to doing this manually, and it's some of my most creative and intellectually productive time. I am contemplating moving to Emacs org-mode for future things, with some sort of conversion process for the accumulated things. That should make a more intelligent system possible.
alistproducer2 11y ago
I was once a warrior on the PK front. I built a PK Chrome extension called Deeper History http://lifehacker.com/deeper-history-searches-the-contents-of-visited-pages-1502340820 http://lifehacker.com/deeper-history-searches-the-contents-o.... It compressed and stored pages you visted in IndexDB. You could then search your history by keyword form your url bar. An example use case was trying to recall a comment of an article you liked. If you could remember a few of the words DH would pull it back up. I stopped developing it once I realized it was a security nightmare. I built a version that encrypted the data before storing it but that ballooned the DB size and I also didn't have a user-friendly way for users to obtain a key set.
ianstormtaylor 11y ago
This is the problem that Dropbox Paper is solving—both for teams and for personal use. I use it now to write all of my blog posts, store research about everyday life things, and keep track of things like doctors, lawyers, etc. I think it solves the problem very well. Early on they hadn't quite nailed "findability", but with the new (more generic) folders ported from the desktop metaphor and "starred" items this is largely solved. And the search is very fast. Not only that, but it makes it incredibly easy to share with friends and even have them comment or contribute to individual files, without having to expose everything you write. Learning curve is near-zero. All in all, super impressed.
massysett 11y ago
I'm surprised he does not mention email. This is my personal knowledgebase. It keeps boatloads of receipts and travel reservations from years ago. It is available on all my computing devices. It's Gmail so it's easy to search and I don't have to spend time sorting it. I write notes and send them to myself just so I will have them in the email. Other people are also obsessive about keeping their old email so I can't possibly be alone in this regard. Gmail nails most of the features on his list.
- jdmichal 11y ago
  I set up a filter on my Gmail that automatically tags emails from myself with "Notes". Extremely easy to add a note anywhere -- just email myself from my phone. Easy to search -- just add "in:Notes" to the criteria. Only thing it doesn't really cover is linking between notes. Though I wonder exactly how valuable that is anyway, when you have a powerful search available.
- hollerith 11y ago
  Do you know how to go from a message to the message's "parent" (the message that the message is a reply to)? I can't figure out how to do that in gmail.
- veidr 11y ago
  Last I checked, Gmail couldn't search the contents of attachments. That is a pretty show-stopping limitation for a "knowledge base"... have they fixed that?
stealthascope 11y ago
A couple years ago, I decided to setup a MediaWiki installation, and found it works well enough for storing thoughts/ideas/things. I added another sort of 'navigation' layer that makes sense in my head, and wrote a desktop app in python to use my navigation layer instead of the links. Eventually I would like to set it up to scan for links in the various pages, download the content then add it to a job queue to be parsed into a more concise format for the wiki itself. I don't use it nearly as much as I should... it's one of those things that's 90% what I want, and just missing the last 10%.
limeyy 11y ago
Anyone Remember Neonem on Windows? Well, that was really good for organising personal info in a structured way. Discussion here seems too broad: comparing Evernote (which is basically losely structured), wiki's etc to real classic outliners that once were thriving on Windows in he old days but kind of have of have been forgotten the past years. If you search for "outliner" you'll find some good all abandoned Windows projects.
siavosh 11y ago
Co-founder of Faqt, mentioned in the blog post. The lack of good options is what drove us to create our own solution, with our primary/religious focus being on simplicity. At the risk of self-promotion, we have been quietly beta testing with several organizations a collaborative/team version. If you'd like to try it, there's a free signup at https://app.faqt.co/team/signup https://app.faqt.co/team/signup Let me know if anyone has any question: hello@faqt.co
mapgrep 11y ago
I don't think he's talking about a personal knowledgebase, I think he's talking about something else we are very familiar with. Think about it, he says a PK is "any system that you use to store and retrieve general information Unlimited size... just about everything you want to save for your whole life." That's a computer, full stop. Or, in software terms, an operating system. In storage terms, a filesystem. These high level tools are designed precisely for the tasks he outlined: Handle data of ever increasing size reliably; make it easy to navigate and search; be simple to use, convenient and fast, and structured. Where operating systems and filesystems fall down is in searching heterogenous data in a powerful way, and that's where people turn to PKs. Evernote is/was good at searching text and even pictures of text and for tagging; "the Brain" is apparently good at cataloging structured to-do and project type data; Kumu seems to do social graphs; inforapid seems to specialize in process descriptions. There is no reason operating systems can't get better at processing this data natively. Apple has slouched in that direction in fits and starts with projects like Spotlight and Quick Look. Microsoft once fantasized about building a filesystem from a relational database and has developed an interesting system of "search contracts" for Windows.
- zzzmarcus 11y ago
  OP here. I mostly agree with you. If the filesystem was better at a few things then it'd be the ideal PK. Right now though it isn't enough. It's too slow or too inconvenient for most things you'd want in a PK. Creating files isn't fast or easy. Usually you need to open some other app to do it, even if it's just a terminal. Linking one file to another and showing the connection with metadata like the direction of the link is possible, but not easy, especially if you want multiple links to and from a file. You can sync with things like Dropbox, but search is painfully slow and viewing/editing files means opening a different app. It's just cumbersome. The reason I like TheBrain is actually because it's good for much more than todos and project data. I have one "file" that has over 10 thousand notes in it. It's organized arbitrarily by my preference, but definitely structured, searchable and "infinitely" expandable. The best part though is that it's really easy to visualize related data in TheBrain. In fact, as I've read comments today, I've come to realize that that is the one crucial thing most PK's are missing. There aren't good visualizations for Evernote, OneNote, most filesystems, org mode, etc. I"m with you, there's no reason the filesystem can't become the best PK, but it's far from that now.
dangoor 11y ago
Chris Granger's latest experiments with Eve (github.com/witheve) look surprisingly like a semi-structured personal knowledge system. Take a look at the last 15 minutes or so of this video for a demo: https://www.youtube.com/watch?v=VZQoAKJPbh8 https://www.youtube.com/watch?v=VZQoAKJPbh8 (the whole video is interesting for the evolution of it)
xenihn 11y ago
Quiver in my iCloud is the best thing that I've found so far. One of the biggest things for me is support for code snippets /highlighting, without having to use something to convert to RTF in-between copy and paste. Sadly, I stopped using it since I now have to log in to my company iCloud and Apple accounts at work, and it's just too much of a hassle to switch back and forth. I mainly use OSX notes and Textmate now.
utopiah 11y ago
several instances of PmWiki + Vimperator, being using that for nearly 10 years and thousands of edits so far
lewisl9029 11y ago
I just came across this a few days ago on HN: https://youtu.be/VZQoAKJPbh8?t=2818 https://youtu.be/VZQoAKJPbh8?t=2818 https://news.ycombinator.com/item?id=10726536 https://news.ycombinator.com/item?id=10726536 Eve, the new "programming" tool developed by Chris Granger of Light Table fame, looks absolutely amazing for this use case. I personally can't wait to try it out.
Razengan 11y ago
I use Markdown notepads like Quiver [1] and Ulysses [2] to get a "pretty view" on my existing plaintext hoard and directory structure. This lets my data remain portable and still gives me (almost) all the niceties of proprietary formats/editors. [1]: http://happenapps.com/#quiver http://happenapps.com/#quiver [2]: http://ulyssesapp.com/mac/ http://ulyssesapp.com/mac/
arvind_devaraj 11y ago
An automated tool for capturing and sharing knowledge will truly make the world a better place. Learning can be faster , reducing the cost of education. People across the globe can connect based on topics of interest and knowledge, yielding more economic and job opportunities. A personal knowledge base is far more valuable than what it might seem at first glance. Very happy to see you have articulated clearly and made people realize it. A PKB has great potential for knowledge transfer within organization or even at research labs for new people to familiarize with literature. Many people want to capture their thoughts for writing a book someday, but current tools aren't good enough for this. Sadly, a person's social graph is well captured today rather than their knowledge graph. With these thoughts, I started implementing Hyperbook - www.getbook.co few months back. Currently it is in beta and really working hard to improve it every day. We got good feedback and reviews from thought leaders in content curation and knowledge management industry. Thanks for bringing the problem for discussion. It gives me motivation that is very much a worthy problem to solve.
Treycent 11y ago
Knowledge can be captured in wikis, notes, documents, videos, audio, and even photos. Our smartphone/smartwatch/web apps allow you to access any of the knowledge using a single, easy to use interface - voice. Basically, you tag content with a voice command (e.g. "to do list" or "show me the video on how to do xyz") then, whenever you need to retrieve the knowledge, just speak the command. Voice commands can be shared and the apps are all free. https://youtube.com/treycent/search?query=notes https://youtube.com/treycent/search?query=notes
anotheryou 11y ago
I'm using: - markdown - + folders (synced to my mobile) - + a flat-file cms to wrap it up in the browser (in my case: yellow cms, but there are a few based on markdown and I could switch any time) Still problematic: - fast and easy linking - suggested tags (no body likes tagging manually) - rich media (especially as you'd ideally want a local mirror, like some research tools do it) and the general problem: - text is linear - folders mostly hierarchic - information is ribosomal - ribosomal connections are rather associative and therefore manifold - manifold structures are a pain in the ass to do manually by linking and tagging An even better google would be the solution: - you just dump information (+ meta data to give some more context and spare you explaining the current event surrounding the note) - and it will be found on demand by AI maybe in a few years I can feed something my wiki and it will make sense of it

115 points by zzzmarcus 11y ago

Amen. I currently use txt files & Tree 2 (mac app). I’ve tried Quiver, Quip, & Workflowy and cannot find the perfect app to suit this space for my usage patterns. I’m almost 100% back to text files & sublime text.

dheera 11y ago
The problem is really that every time something new and better comes around, it becomes a mess to port everything over, especially if it is conceptually different (e.g. tag-based or something else). Dying products also seldom give you a convenient means to export your content. So ultimately the only thing that can last for decades, be cross-platform, and not be tied to one company's product, is text files and directories. For now, throw it all on Dropbox, Github, or whatever you like. When Dropbox and Github die, the file-directory structure will be easy to port to whatever is next.
- daviross 11y ago
  This has been my current approach for a while, after exporting everything out of Tiddlywiki when it looked like that was dying for a while (Browser support drying up, etc). Now that it's been revived with Tiddlywiki5 and development's picked back up, I'm somewhat tempted to get back into it. Main advantage: Better crosslinking, rich content, etc. Main disadvantage: Less simple to edit, some tedium to export. I'd love to have a Tiddlywiki-like frontend to a set of Markdown text documents backing it. (This is something that's been gnawing at the back of my mind long enough that I might have to buckle down and make it myself) Maybe Markdown + JSON for rich content data...
  - dheera 11y ago
    Where I need markup, I've been sticking to HTML or LaTeX syntax for the most part. Partly because I hate Markdown due to its inconsistent syntax (that's my personal opinion), but also because it feels like a fad of recent years, much like a successor of the MediaWiki fad and the BBcode fad. I feel like in another 2 or 3 years someone will invent Markleft or Markright or Visual Markdown#.NET 2.0 or Dustindown that's 10 times better than Markdown, and then I'll have to switch everything over yet again. HTML and LaTeX on the other hand, aren't dying any time soon due to their extensive deployment, and I'm confident that my syntax will be renderable for at least another 10 years or more.
    - hammerandtongs 11y ago
      Emacs and orgmode are probably your next step :) orgmode is really a superset of markdown with PIM markup added. It exports to markdown and a ton of other formats. Cheers.
    - ta0967 11y ago
      i'm not sure you'd consider it better markdown (i do), but do you know about reStructuredText? it seems to even predate markdown. http://docutils.sourceforge.net/docs/user/rst/quickstart.html http://docutils.sourceforge.net/docs/user/rst/quickstart.htm...

nwatson 11y ago

I like Evernote for a lot of personal stuff, though that's one of the "worser" systems OP lists. Atlassian's Confluence does very well at being a wiki and though you're hostage perhaps to your original layout / organization, at $10/year for self-hosted it's a very good deal. It can store and index PDF, Word, Excel, etc., documents very well. I don't think it does handwriting recognition. If I were to evolve the product category I'd use Confluence as a basis. In fact I'd want to toy with using Confluence (+ Postgres backend) as the basis for any I.T.-related product -- write plugins for user input, generate all reports to Confluence. I'm not sure what licensing restrictions might be but Confluence provides a lot of useful UI for $10/year (structured + unstructured) for any customer, and building on top of that seems like a no-brainer. Perhaps this is what others do with Drupal and other CMS products, though ... I'm not familiar with that model.

djur 11y ago
I'm surprised to read this. Confluence is the most unpleasant "wiki" product (its lack of support for redlinks prevents me from really considering it a wiki) I have ever used. Its built-in hierarchical structure for pages is excessively intrusive and undercooked at the same time, and the search is so bad that I have had it fail to pull up pages when I typed in the literal, complete title. Like most Atlassian products, its only apparent notable quality to me is that it integrates with other Atlassian products, as well as being a semi-functional platform on which to pile plugins and customization. It's possible that there's a good Confluence install out there, probably administered by the same people who run that really nice JIRA instance that people claim to have used once.
- eponeponepon 11y ago
  Confluence is... okay. I think Atlassian have successfully managed to sand down a lot of the sharp corners on older wiki-type packages that tend to act as a barrier to the non-technical types - from my perspective, that seems to mean that my non-technical colleagues can be persuaded to contribute to documents held in Confluence markedly more easily than to contribute to a MediaWiki instance that we previously had. So, while Confluence itself as a piece of software makes me grumble more often than not, I'm more than happy to trade that off against the easier sell to others.

herge 11y ago

My saving grace at work has been the tidily wiki (http://www.tiddlywiki.com http://www.tiddlywiki.com). It's the easiest place to put one-off shell command line snippets, or instructions or whatever. It's search is fast, and the formatting is nice. The only downside is I have to remember to save and copy the downloaded file to google drive every time I edit anything.

ahnick 11y ago
Yeah tiddlywiki is by far my preferred way of storing this type of information. Coupled with a FreeNAS box I can't really think of a better way to store this type information.
- eponeponepon 11y ago
  I never managed to get tiddlywiki working right - pity, because I really wanted it to work for me, but couldn't seem to get to a stage where I'd trust it to have saved something when it said it had. This was five years ago at least, mind - I've no idea what the problem might've been, and even if it wasn't user error, I suppose it might've evaporated by now. (note to self: try tiddlywiki again soon)
fsiefken 11y ago
Have you tried the firefox save plugin? You could use it to save to a synched google drive so you don't have to think about that.
cmrx64 11y ago
I love love love tiddlywiki. I've been using it for probably 6 or 7 years now. It's simply amazing.

cdumler 11y ago

I commented on Discus the site, as well. I've been a long time user of DEVONthink. It can be used for many purposes, including document organizer, email, GTD, tagging, etc. My favorite feature has always been the Magic Hat, which will suggest under what folders a document should be filed. I clip most interesting web pages, and it is surprisingly accurate once you have established a few documents. http://www.devontechnologies.com http://www.devontechnologies.com

look_lookatme 11y ago
I agree, it's great. I just wish DEVONthink had a better iOS client.

bachmeier 11y ago

In my opinion, this is not a "problem" that can be solved. What makes something good varies a lot from one person to the next. In my case, I'd need a way to hold equations, computer code, PDF versions of papers, handwritten notes that someone else has given me, and a lot of other things. I want it stored in a Git repo. I want to view it in a browser. I want it to be convenient and I want it automated. The "answer" is a feature-filled scripting language that allows you to build a customized solution.

civilian 11y ago
What about it would you want automated? What kind of scripts would you add to your personal knowledgebase?
- bachmeier 11y ago
  It's not so much what I would want automated as what I have already automated. As an example, consider how I handle notes that I leave for myself as I'm working. These are things like paragraphs to be added to a paper, ideas for a research topic, a list of things I have to do around the yard, or anything not directly related to the task at hand. I have written a script in D (my programming language of choice). I have a web interface to all of the notes I've jotted down in the past. I can click a topic and see all notes related to that topic. I click the new note button, and it opens Emacs with the file that will hold the note. I type in a note in markdown, and as soon as I close Emacs, the note is converted to html. If I want to change it, I hit the edit button, Emacs opens up, and I make my edits. Upon closing Emacs, the html is rebuilt. Anytime I create a new note or edit an existing note, there is a commit to the Git repo. That's what I mean by automated. I write scripts that handle everything except the content of the notes. I have written or am writing scripts for just about everything I do.

cypherg 11y ago

Confluence for the simple win

irln 11y ago

I've found that using google docs works well if you frame the document title in the form of a question or statement: - What is a JavaScript constructor? - making rvm permanent in rvspec - postgresql listing, creating and dropping a db Some of the "answers" are just a single line. Then all I have to remember is either something in the title or the gist of the thing I need to know and google docs search provides me with a list of related stuff.

pullo 11y ago

Evernote. Free and accessible across many devices. For me it has been best of the worst. It has been my one place to store all information. Evernote serves well as a general knowledge base, but fails for storing programming snippets and ideas.

oxplot 11y ago

How about Camlistore by Brad Fitzpatrick [1]: "Camlistore is a set of open source formats, protocols, and software for modeling, storing, searching, sharing and synchronizing data in the post-PC era. Data may be files or objects, tweets or 5TB videos, and you can access it via a phone, browser or FUSE filesystem." [1]: http://camlistore.org/ http://camlistore.org/

pjc50 11y ago
Camlistore is a nice append-orientated sync system, but I wasn't aware it included any data modelling or search?
epaulson 11y ago
Camlistore is where I'm expecting to go. They've got some basic things modeled in their world, and the rest I'll fill in using inspiration from schema.org. My plan is to land PDFs of research papers, web bookmarks (and archives of those pages), ebooks, book scans, textfiles/markdown documents, imports of social networks (favorites from Twitter, mostly) and scans of written notes I make to myself in Camlistore. From there, sync to devices and publish to the web as appropriate. I don't expect that I'll ever find the time to build this all myself, so I'm hopeful that either my data will be in a format that other Camlistore apps can read, or if I get the format wrong I'll be able to write a translator to whatever the Camlistore community decides to model for that specific data type.

gkya 11y ago

Org-mode. A benevolent beast. Once one gets to learn it, it can be made to do anything. Also having a programming language shared with the runtime and other programmes for that runtime (Elisp and Emacs) is just priceless, it just lends itself to be customized in whichever way one would like. A major con for me, though, is the lack of decent iOS-Android integration.

kristiandupont 11y ago
I've been wanting to learn org-mode for years but I never get around to it. I use something similar, but I only worry that if I were to move to something else I would want it to be in the cloud so I could access it from devices and other laptops. Do you have a solution for this?
- eeZi 11y ago
  It's just plain text files, so you can sync it using whatever you want. Many people use Dropbox.
- spatten 11y ago
  I have a bunch of org-mode files that I save into a Dropbox folder. That gives me backups, and access from other computers / laptops. Search is via ack or grep. It theoretically also gives me access from my phone using MobileOrg[1], but I have never actually set that up. [1]: http://orgmode.org/manual/MobileOrg.html http://orgmode.org/manual/MobileOrg.html
  - gkya 11y ago
    > ack or grep C-c C-j. You're missing out :)
    - spatten 11y ago
      Cool, didn't know about that. Does it search multiple files in a directory? I use it for work logs and keep one file per week.

Alex3917 11y ago

In my opinion, FreeMind is the best solution currently out there because of forces mindmapping conventions and places a strong emphasis on folding. A lot of other solutions I've seen are more like concept maps, which are great for project management, but not great for knowledge bases. http://freemind.sourceforge.net/wiki/index.php/Main_Page http://freemind.sourceforge.net/wiki/index.php/Main_Page (The one thing of note is that there is a bug in the default version of OS X Java, so it frequently crashes unless you update to Java 8 and edit the config to use that version of Java.)

zzzmarcus 11y ago
Freemind is great for mindmapping in the context of a specific topic or project, but isn't a general purpose knowledgebase. A mindmap starts at a central node and moves outward. This makes it limited in the practical number of nodes you can have--anything over a couple hundred is hard to manage. It also makes it difficult for one node to appear in multiple places within the mindmap.
jpfed 11y ago
>A lot of other solutions I've seen are more like concept maps, which are great for project management, but not great for knowledge bases. I've never understood this preference; would you be willing to explain it? It seems almost exactly backward to me: I model tasks as trees of sub-tasks, and knowledge as a general graph of connected concepts.
- Alex3917 11y ago
  So all knowledge can be seen as connected, but all knowledge can also be put into hierarchies. Concepts and how those concepts are related is something that people come up with, rather than being an inherent property of the universe. The idea of forcing yourself to put knowledge into hierarchies is that it makes your thinking more clear, makes it easier to remember where things are, and perhaps paradoxically makes it easier to see how things are related. (Or at least highlights certain types of relationships that are only visible in the context of hierarchies.) I'll admit it's pretty weird at first to only get one root node, I think it took me a couple years to get used to that. But once I embraced it the benefits quickly became clear.

space_ghost 11y ago

I've been using Zim (http://zim-wiki.org http://zim-wiki.org) for several years, backed up to Dropbox. It's a significant improvement over plain text files.

pjc50 11y ago

OneNote?

douche 11y ago
I've become a big fan. With the OneNote clipper extension for Chrome, it's how I bookmark everything I might want to read later.

msluyter 11y ago

I've been using nvAlt for a while. Very fast search and ease of use. Not sure how well it scales in the long run but thus far I've had no problems. Edit to add: Some other interesting features. You can sync to Simplenote automatically, and notes can be saved as text files for easy import/export. There is some markdown support (though I see a number of bugs in the preview). The main downsides appear to be: limited color schemes, no syntax highlighting for markdown, development appears to have stalled.

zwischenzug 11y ago

I have a git repo with notes on all the things I'm interested in, a few larger files (some books I always reference, some images etc.). Over time I've built up some scripts to help with general tasks, eg list TODOs etc.. It's perfectly moulded to my needs. For example, when I find a new utility I run mk_notes.sh [thing], which sets up three files: - links (for relevant web links) - thing.txt (which has basic info about it) - cheat_sheet.txt (which has quick useful things I want to keep) These files suffice for the majority of cases. I used to have more, but found that was over-engineered. I'm not sure I could buy such a thing, and if I did, I'd get irritated with some aspect of it sooner or later. Search is easy (grep et al), git means it's available everywhere (for me). It's literally changed my life, since I now don't 'lose' any research, and can pick up threads later and build.

eeZi 11y ago
That sounds a lot like the org-mode concept.
- zwischenzug 11y ago
  Interesting; a quick glance at the home page, and it looks like it's for planning. I'm happy using JIRA for that, where I have customized workflows. But again, this is just because I'm familiar with JIRA, not because it's necessarily the best tool. Also, I'm a vi user :)
  - eeZi 11y ago
    That's not mutually exclusive - see https://news.ycombinator.com/item?id=10739693 https://news.ycombinator.com/item?id=10739693 Most people use it for planning, but it's also a very nice personal wiki. The great thing is that you can define your tasks inline, if you want.

PaulHoule 11y ago

I use Microsoft's OneNote for this. The search works amazingly well and is fast.

howeyc 11y ago
I've been tempted by this, but I have some reservations: 1. Does it work under less-used OSs? Linux/BSD..? 2. Is it a proprietary format? Will you be able to open the file in a decade? My text files will be able to be opened, I'm not sure anyone can still open their MSWorks files. Maybe I'm crazy to worry about opening these files after a decade since using them probably isn't worth while unless you keep up with it. However, I've got files from the 90's that I've been transitioning from machine-to-machine since storage costs keeps getting cheaper.
- eeZi 11y ago
  No, and yes. It's proprietary and Windows/Android/iOS-only.
  - falcolas 11y ago
    Toss Mac in there as well: https://itunes.apple.com/us/app/microsoft-onenote/id784801555?mt=12 https://itunes.apple.com/us/app/microsoft-onenote/id78480155...
  - PaulHoule 11y ago
    That's false, you can download documentation for the OneNote file format here https://msdn.microsoft.com/en-us/library/dd924743(v=office.12).aspx https://msdn.microsoft.com/en-us/library/dd924743(v=office.1... it is only about 5% of the size of the Adobe PDF specification and I am sure there are corner cases, but it is not too bad.
    - eeZi 11y ago
      I admittedly did not know that they published a specification . It's still a proprietary file format though. Proprietary does not mean "binary and undocumented", it means that it's not an open standard.

howeyc 11y ago

Any reason a bunch of text files in directories wont work? Any "advanced" text editor that can search directories should work: Notepad++,Vim,grep,... anything really. I've been using a bunch of synced markdown files. Works for me, I just have a Markdown editor and a "grep-like" app for android. I'm not sure what you can do for iPhone though. When I've talked to people they couldn't seem to "browse for files/folders" which I thought was strange. The one thing I'm missing is I'd like to "tag" my notes. I haven't decided how/if I'm going to do that yet.

EC1 11y ago
I use this [1] to do a simple full-text search of my personal knowledge base. [1] https://github.com/slashdotdash/jekyll-lunr-js-search https://github.com/slashdotdash/jekyll-lunr-js-search
zzzmarcus 11y ago
One of the main reasons for using a personal knowledgebase is surfacing patterns in what you've saved. If there is no way of visualizing the relationship between notes/nodes outside of a flat list, pattern finding is limited to what you can remember to search for. Basically the core problem isn't saving and searching for data, it's having great ways to structure and visualize it.
- howeyc 11y ago
  Good point, unless I know a search term it doesn't work out great. I've been thinking of switching to some kind of wiki format, like maybe creole. Then finding an app that can understand links between files or something.

newman314 11y ago

I used to use Delicious but that didn't solve the "search the content of my bookmarks" problem for me. I've been using Pocket for now and it seems to work fine.

hackercomplex 11y ago

imho the best one ever made is called getguru.com. It's brilliant actually but the team there has priced themselves out of the market charging $7/head as a b2b app because they're from and enterprise sales background and therefor don't realize guru's monster potential in consumer-web. They're happy taking the low hanging fruit $$ from call-centers. sigh

zeveb 11y ago

I've gotta say that org-mode and a git repo do pretty well by me for this kind of thing.

dalacv 11y ago

I like trunknotes (http://www.trunknotes.com/ http://www.trunknotes.com/)

eponeponepon 11y ago

I'm inclined not to see this as surprising. There are as many ways in which people conceptualise their 'offline' knowledge as there are people, and there is never going to be one size to fit all. Org-mode is probably the closest contender, if only for its sheer flexibility - but the fact that it's wed to something (i.e. emacs) that the gen-pop would immediately recoil from is only ever going to limit its adoption to a tiny subset.

biztos 11y ago

I think vendor lock-in is at least as big a problem as searching and scaling. Arguably bigger for this audience: if I have easy unfettered access to the data itself I can always try something else for search/scale. I've tried a few and at least twice tried to write my own (and gave up due to scope creep). Right now I just have a bunch of Markdown files and a giant messy DropBox folder and an almost-as-messy set of private GitHub repos. And photos. And videos. And e-mail. And backups. (What does one do with a Zip disk these days?) I bet a lot of people my age (mid-40's) or older have the same problem: choosing or building a system we think would work long-term, and then gathering and organizing all the "legacy" stuff (which you definitely want!) is already an enormous undertaking, and so instead we live with a bunch of ad-hoc systems that aren't in any way cross-referenced. Nobody wants to be the sysadmin of their own junk drawer, but I for one don't want years' worth of my stuff locked into somebody's Cloud du Jour. (Then again, I have this same problem with actual paper documents, so maybe it's just me.)

xbryanx 11y ago
This. As a near 40 person, I now need to search in 5-10 places for that thing I know I wrote down in Markdown, Vim Notes, Workflowy, Tiddlywiki, 1Password, Confluence, or email at some point in the last 20 years. But maybe we just need to give up on finding one tool to store them all. Instead we need to build more federated search across all these platforms. That way I can have one searcher that returns results all my various old "brains."
dangoor 11y ago
I have had very similar thoughts, but I have to wonder if it turned out to be wrong. Jerry's Brain: http://jerrysbrain.com/ http://jerrysbrain.com/ Apparently started in TheBrain in 1997 and now has more than 250K "thoughts". That's breathtaking for both the longevity and the scale. I'm guessing that most mind mapping programs wouldn't handle that well. Gathering the legacy stuff is not as important as getting started with something, I think. The legacy will make it into a new system as it's needed. I've been using Evernote for a number of years. I don't particularly like it (cumbersome UI without great navigation), but the search has worked well enough for me to find things and its ubiquity has been great.

pfista 11y ago

Does anyone have a system for managing their relationships? Kind of like CRM but on a personal level. Keeping track of people's details - like their family, their passions, their history etc.

deleted 11y ago

[deleted]

eeZi 11y ago

I, too, am an org-mode fan, but there are a few good ones missing: - Zim (http://zim-wiki.org/ http://zim-wiki.org/, Open Source) - WikidPad (http://wikidpad.sourceforge.net/ http://wikidpad.sourceforge.net/, Open Source) - ConnectedText (http://www.connectedtext.com/ http://www.connectedtext.com/, Windows-only, $$$)

karmacondon 11y ago

I've been working with own "PK" for about a year now. It's been an interesting experiment so far. My concept is based around random re-exposure to information. For whatever reason, the thought of "Let me look through my old bookmarks" never occurred to me and wasn't part of my daily workflow. So one aspect of my system is that it randomly selects bookmarks from different categories every day and shows them to me. It also has spaced repetition for facts and miscellany, or things I'm trying to learn about (like the US Presidents or Shakespeares plays), vocabulary, etc. There were a few interesting things I've noticed by randomly viewing old bookmarks. The first is that linkrot is a very real thing and the web is decaying at a measurable rate. Second, it's amazing how limited my memory is. Half the time I don't remember something I bookmarked from years or even months ago at all. It's new to me, all over again. I'm going to bookmark this discussion and then completely forget it existed within a few months at most. Regularly reviewing old web pages and notes makes me feel like my mind is a sieve, as well as highlighting how my interest and thought processes have changed over the years. I'm really interested in this topic, feel free to contact me to discuss.

bovermyer 11y ago
That's an interesting concept. Is the selection of random bookmarks completely random, or does it use a more complex algorithm to choose what to display?
- karmacondon 11y ago
  There are simple rules, like "Don't show the same bookmark again within X months", where X becomes progressively longer the more times the bookmark has been shown. I can also flag the best stuff so it shows up more often. I did my best to group the text of the urls, using first kmeans in php (which is a very stupid idea) and finally settling on LDA. I'd like to use hierarchical LDA to automatically discover topics within my bookmarks, but that's "Future Work" for now.
djhn 11y ago
Sounds like what I have been doing with Evernote. I described the system in a previous discussion: https://news.ycombinator.com/item?id=10301205 https://news.ycombinator.com/item?id=10301205 What you say about review resonates with me. Several years ago I got excited about Spaced repetition software (like Anki, discussed recently on HN) and tried writing an Evernote app that would periodically show me interesting things. Their API didn't work very well for this, so I stuck to doing this manually, and it's some of my most creative and intellectually productive time. I am contemplating moving to Emacs org-mode for future things, with some sort of conversion process for the accumulated things. That should make a more intelligent system possible.

alistproducer2 11y ago

I was once a warrior on the PK front. I built a PK Chrome extension called Deeper History http://lifehacker.com/deeper-history-searches-the-contents-of-visited-pages-1502340820 http://lifehacker.com/deeper-history-searches-the-contents-o.... It compressed and stored pages you visted in IndexDB. You could then search your history by keyword form your url bar. An example use case was trying to recall a comment of an article you liked. If you could remember a few of the words DH would pull it back up. I stopped developing it once I realized it was a security nightmare. I built a version that encrypted the data before storing it but that ballooned the DB size and I also didn't have a user-friendly way for users to obtain a key set.

ianstormtaylor 11y ago

This is the problem that Dropbox Paper is solving—both for teams and for personal use. I use it now to write all of my blog posts, store research about everyday life things, and keep track of things like doctors, lawyers, etc. I think it solves the problem very well. Early on they hadn't quite nailed "findability", but with the new (more generic) folders ported from the desktop metaphor and "starred" items this is largely solved. And the search is very fast. Not only that, but it makes it incredibly easy to share with friends and even have them comment or contribute to individual files, without having to expose everything you write. Learning curve is near-zero. All in all, super impressed.

massysett 11y ago

I'm surprised he does not mention email. This is my personal knowledgebase. It keeps boatloads of receipts and travel reservations from years ago. It is available on all my computing devices. It's Gmail so it's easy to search and I don't have to spend time sorting it. I write notes and send them to myself just so I will have them in the email. Other people are also obsessive about keeping their old email so I can't possibly be alone in this regard. Gmail nails most of the features on his list.

jdmichal 11y ago
I set up a filter on my Gmail that automatically tags emails from myself with "Notes". Extremely easy to add a note anywhere -- just email myself from my phone. Easy to search -- just add "in:Notes" to the criteria. Only thing it doesn't really cover is linking between notes. Though I wonder exactly how valuable that is anyway, when you have a powerful search available.
hollerith 11y ago
Do you know how to go from a message to the message's "parent" (the message that the message is a reply to)? I can't figure out how to do that in gmail.
veidr 11y ago
Last I checked, Gmail couldn't search the contents of attachments. That is a pretty show-stopping limitation for a "knowledge base"... have they fixed that?

stealthascope 11y ago

A couple years ago, I decided to setup a MediaWiki installation, and found it works well enough for storing thoughts/ideas/things. I added another sort of 'navigation' layer that makes sense in my head, and wrote a desktop app in python to use my navigation layer instead of the links. Eventually I would like to set it up to scan for links in the various pages, download the content then add it to a job queue to be parsed into a more concise format for the wiki itself. I don't use it nearly as much as I should... it's one of those things that's 90% what I want, and just missing the last 10%.

limeyy 11y ago

Anyone Remember Neonem on Windows? Well, that was really good for organising personal info in a structured way. Discussion here seems too broad: comparing Evernote (which is basically losely structured), wiki's etc to real classic outliners that once were thriving on Windows in he old days but kind of have of have been forgotten the past years. If you search for "outliner" you'll find some good all abandoned Windows projects.

siavosh 11y ago

Co-founder of Faqt, mentioned in the blog post. The lack of good options is what drove us to create our own solution, with our primary/religious focus being on simplicity. At the risk of self-promotion, we have been quietly beta testing with several organizations a collaborative/team version. If you'd like to try it, there's a free signup at https://app.faqt.co/team/signup https://app.faqt.co/team/signup Let me know if anyone has any question: hello@faqt.co

mapgrep 11y ago

I don't think he's talking about a personal knowledgebase, I think he's talking about something else we are very familiar with. Think about it, he says a PK is "any system that you use to store and retrieve general information Unlimited size... just about everything you want to save for your whole life." That's a computer, full stop. Or, in software terms, an operating system. In storage terms, a filesystem. These high level tools are designed precisely for the tasks he outlined: Handle data of ever increasing size reliably; make it easy to navigate and search; be simple to use, convenient and fast, and structured. Where operating systems and filesystems fall down is in searching heterogenous data in a powerful way, and that's where people turn to PKs. Evernote is/was good at searching text and even pictures of text and for tagging; "the Brain" is apparently good at cataloging structured to-do and project type data; Kumu seems to do social graphs; inforapid seems to specialize in process descriptions. There is no reason operating systems can't get better at processing this data natively. Apple has slouched in that direction in fits and starts with projects like Spotlight and Quick Look. Microsoft once fantasized about building a filesystem from a relational database and has developed an interesting system of "search contracts" for Windows.

zzzmarcus 11y ago
OP here. I mostly agree with you. If the filesystem was better at a few things then it'd be the ideal PK. Right now though it isn't enough. It's too slow or too inconvenient for most things you'd want in a PK. Creating files isn't fast or easy. Usually you need to open some other app to do it, even if it's just a terminal. Linking one file to another and showing the connection with metadata like the direction of the link is possible, but not easy, especially if you want multiple links to and from a file. You can sync with things like Dropbox, but search is painfully slow and viewing/editing files means opening a different app. It's just cumbersome. The reason I like TheBrain is actually because it's good for much more than todos and project data. I have one "file" that has over 10 thousand notes in it. It's organized arbitrarily by my preference, but definitely structured, searchable and "infinitely" expandable. The best part though is that it's really easy to visualize related data in TheBrain. In fact, as I've read comments today, I've come to realize that that is the one crucial thing most PK's are missing. There aren't good visualizations for Evernote, OneNote, most filesystems, org mode, etc. I"m with you, there's no reason the filesystem can't become the best PK, but it's far from that now.

dangoor 11y ago

Chris Granger's latest experiments with Eve (github.com/witheve) look surprisingly like a semi-structured personal knowledge system. Take a look at the last 15 minutes or so of this video for a demo: https://www.youtube.com/watch?v=VZQoAKJPbh8 https://www.youtube.com/watch?v=VZQoAKJPbh8 (the whole video is interesting for the evolution of it)

xenihn 11y ago

Quiver in my iCloud is the best thing that I've found so far. One of the biggest things for me is support for code snippets /highlighting, without having to use something to convert to RTF in-between copy and paste. Sadly, I stopped using it since I now have to log in to my company iCloud and Apple accounts at work, and it's just too much of a hassle to switch back and forth. I mainly use OSX notes and Textmate now.

utopiah 11y ago

several instances of PmWiki + Vimperator, being using that for nearly 10 years and thousands of edits so far

lewisl9029 11y ago

I just came across this a few days ago on HN: https://youtu.be/VZQoAKJPbh8?t=2818 https://youtu.be/VZQoAKJPbh8?t=2818 https://news.ycombinator.com/item?id=10726536 https://news.ycombinator.com/item?id=10726536 Eve, the new "programming" tool developed by Chris Granger of Light Table fame, looks absolutely amazing for this use case. I personally can't wait to try it out.

Razengan 11y ago

I use Markdown notepads like Quiver [1] and Ulysses [2] to get a "pretty view" on my existing plaintext hoard and directory structure. This lets my data remain portable and still gives me (almost) all the niceties of proprietary formats/editors. [1]: http://happenapps.com/#quiver http://happenapps.com/#quiver [2]: http://ulyssesapp.com/mac/ http://ulyssesapp.com/mac/

arvind_devaraj 11y ago

An automated tool for capturing and sharing knowledge will truly make the world a better place. Learning can be faster , reducing the cost of education. People across the globe can connect based on topics of interest and knowledge, yielding more economic and job opportunities. A personal knowledge base is far more valuable than what it might seem at first glance. Very happy to see you have articulated clearly and made people realize it. A PKB has great potential for knowledge transfer within organization or even at research labs for new people to familiarize with literature. Many people want to capture their thoughts for writing a book someday, but current tools aren't good enough for this. Sadly, a person's social graph is well captured today rather than their knowledge graph. With these thoughts, I started implementing Hyperbook - www.getbook.co few months back. Currently it is in beta and really working hard to improve it every day. We got good feedback and reviews from thought leaders in content curation and knowledge management industry. Thanks for bringing the problem for discussion. It gives me motivation that is very much a worthy problem to solve.

Treycent 11y ago

Knowledge can be captured in wikis, notes, documents, videos, audio, and even photos. Our smartphone/smartwatch/web apps allow you to access any of the knowledge using a single, easy to use interface - voice. Basically, you tag content with a voice command (e.g. "to do list" or "show me the video on how to do xyz") then, whenever you need to retrieve the knowledge, just speak the command. Voice commands can be shared and the apps are all free. https://youtube.com/treycent/search?query=notes https://youtube.com/treycent/search?query=notes

anotheryou 11y ago

I'm using: - markdown - + folders (synced to my mobile) - + a flat-file cms to wrap it up in the browser (in my case: yellow cms, but there are a few based on markdown and I could switch any time) Still problematic: - fast and easy linking - suggested tags (no body likes tagging manually) - rich media (especially as you'd ideally want a local mirror, like some research tools do it) and the general problem: - text is linear - folders mostly hierarchic - information is ribosomal - ribosomal connections are rather associative and therefore manifold - manifold structures are a pain in the ass to do manually by linking and tagging An even better google would be the solution: - you just dump information (+ meta data to give some more context and spare you explaining the current event surrounding the note) - and it will be found on demand by AI maybe in a few years I can feed something my wiki and it will make sense of it