3 ms·

On my todo list: cutting up the #clojure freenode logs [1], putthing those paragraphs thru Solr and sphinx. [1] http://clojure-log.n01se.net/date/ http://cloju

by gtani 17y ago

On my todo list: cutting up the #clojure freenode logs [1], putthing those paragraphs thru Solr and sphinx. [1] http://clojure-log.n01se.net/date/ http://clojure-log.n01se.net/date/ http://rottyforge.yi.org/irclogs/freenode/%23clojure/ http://rottyforge.yi.org/irclogs/freenode/%23clojure/ (clojure, #guile, #lisp, #scheme)

gtani 17y ago
on my "todo when i get a sudden infusion of IQ points" list (last time i checked, difficult to impossible in SOLR, sphinx, xapian, tsearch): - easier search on scala, clojure, JS, erlang line noise: "_+_", ">:", "::" etc. - tho with sphinx 0.9.9, thinkikng sphinx, it's easy to do multiple indexes on one rails model, need to revisit..
- gtani 17y ago
  O'reilly has released the entire content of a few (really good) books under Open Feedback: http://labs.oreilly.com/ofps.html http://labs.oreilly.com/ofps.html including the scala, haskell, couchdb and mercurial --------------- http://www.freetechbooks.com/functional-programming-f34.html http://www.freetechbooks.com/functional-programming-f34.html ------------- actually, couchdb book isn't mentioned, it's here http://books.couchdb.org/relax/ http://books.couchdb.org/relax/
- kordless 17y ago
  Check out http://code.google.com/p/semicomplete/wiki/Grok http://code.google.com/p/semicomplete/wiki/Grok. It might be able to parse incoming stuff enough you could tag them with categories.
epi0Bauqu 17y ago
I use solr for the Wikipedia paragraph index. Let me know if you want to know any more about my setup. Also, if it turns out well, I'd love to add this source to DDG :)
- gtani 17y ago
  I'd like to know how you do precision/recall testing. I've spent a lot of time doing pairwise testing of SOLR, sphinx and ferret (haven't tried Xapian), checking which hits one indexer misses that the other gets, stopwords (increments of 0, 50, 100, 175), combinations of token separators, etc. I don't really have to worry about UTF8 to ascii conversion with my data. I haven't really gotten a rigorous methodology or a good number that says precision and recall are acceptable, aside from lists of (a few hundreds of) test queries per topic where i record numbers of hits i expect.
  - epi0Bauqu 17y ago
    You mean the precision/recall evaluation process or specific tweaks within solr itself to achieve it? The former is like you, through test cases, though I have real data streaming in all the time. The latter is incremental, but includes multiple fields and boost functions, as well as query and result manipulation.

by gtani 17y ago

gtani 17y ago

on my "todo when i get a sudden infusion of IQ points" list (last time i checked, difficult to impossible in SOLR, sphinx, xapian, tsearch): - easier search on scala, clojure, JS, erlang line noise: "_+_", ">:", "::" etc. - tho with sphinx 0.9.9, thinkikng sphinx, it's easy to do multiple indexes on one rails model, need to revisit..

gtani 17y ago
O'reilly has released the entire content of a few (really good) books under Open Feedback: http://labs.oreilly.com/ofps.html http://labs.oreilly.com/ofps.html including the scala, haskell, couchdb and mercurial --------------- http://www.freetechbooks.com/functional-programming-f34.html http://www.freetechbooks.com/functional-programming-f34.html ------------- actually, couchdb book isn't mentioned, it's here http://books.couchdb.org/relax/ http://books.couchdb.org/relax/
kordless 17y ago
Check out http://code.google.com/p/semicomplete/wiki/Grok http://code.google.com/p/semicomplete/wiki/Grok. It might be able to parse incoming stuff enough you could tag them with categories.

epi0Bauqu 17y ago

I use solr for the Wikipedia paragraph index. Let me know if you want to know any more about my setup. Also, if it turns out well, I'd love to add this source to DDG :)

gtani 17y ago
I'd like to know how you do precision/recall testing. I've spent a lot of time doing pairwise testing of SOLR, sphinx and ferret (haven't tried Xapian), checking which hits one indexer misses that the other gets, stopwords (increments of 0, 50, 100, 175), combinations of token separators, etc. I don't really have to worry about UTF8 to ascii conversion with my data. I haven't really gotten a rigorous methodology or a good number that says precision and recall are acceptable, aside from lists of (a few hundreds of) test queries per topic where i record numbers of hits i expect.
- epi0Bauqu 17y ago
  You mean the precision/recall evaluation process or specific tweaks within solr itself to achieve it? The former is like you, through test cases, though I have real data streaming in all the time. The latter is incremental, but includes multiple fields and boost functions, as well as query and result manipulation.