3 ms·

Automatically generating a concept index for freeform notes

• chime 16y ago
  This is very similar to something I did few years ago with 6-7 years of my blog entries. I wrote a script that generates timeline-based tag-clouds from plain-text: http://chir.ag/projects/tagline/ http://chir.ag/projects/tagline/ and here's an example: http://chir.ag/projects/preztags/ http://chir.ag/projects/preztags/ The basic algorithm is nearly the same and it does use stemming (though not synonyms, just related spelling). It takes an XML input file and spits out an HTML file with the required JS embedded.
  • LiveTheDream 16y ago
    The presidential speech example is really interesting. Do you have that cleaned-up dataset?
• samstokes 16y ago
  Worth a read for the vector maths interpretation: I could calculate the vectors for all of my notes and use something like the k-means algorithm to find semantically-related clusters of notes. If you're familiar with information retrieval techniques, there's probably nothing new here, but eye-opening if you're rusty like me.
  • thomas11 16y ago
    Look up Latent Semantic Indexing (LSI) on how to group related terms to concepts.