3 ms·

* No decentralized, redundant, internet archive. Solution? I dunno. I've been banging away at it for years. * No easy way to search my personal web/reading his

by aristus 17y ago

* No decentralized, redundant, internet archive. Solution? I dunno. I've been banging away at it for years. * No easy way to search my personal web/reading history. The problem is not just collection, it's relevance. I want to go back and find a book recommendation on design I saw in a blog post sometime in the last week. I dunno where it is or if I bookmarked it. Solution? Google Desktop / WebMynd plus some radically different ideas about search. * I can't use Amazon EC2 with Mosso CloudFiles quickly, cheaply, and at scale. Solution? Cloud peering.

jimmybot 17y ago
I've thought about number two as well. How about a combination of caching all pages you read and how long you are spending on each page (and maybe if you scrolled as well)? The idea is to save enough data that you can have an algorithm that will automatically differentiate between stuff you closed within a couple of seconds and didn't really read and stuff that interested you and you actually read through. And the reason for doing this automatically is it seems very difficult to have the discipline take an action every time you see a page that you may want to save so that you can search through again (such as clicking a box for every such page). Whatever mechanism it is, it needs to account for stuff read through readers and over multiple computers/devices too. I don't want to miss a search result just because I was reading it on an iPhone or in my Google Reader. Or instead of caching locally, a search interface that would accept a long list of urls to look through.
- ntoshev 17y ago
  I have thought about it as well. The signals that could determine relevancy in your browser history are likely different form the ones that determine it for web search, and measuring engagement with a page is a good example (it would help web search as well, but it would be harder to measure). Other things you'd take into account are: * continuity in time - reading several pages on similar topics one after the other. * analysis of the subgraph of the web graph that you visited * you'd be more aggressive with query expansion because you work with much smaller corpus, e.g. you'd want to take hypernyms and hyponyms into account
  - jimmybot 17y ago
    Ah yes, engagement is the word. Interesting. I'd imagine the query expansion should also work across the author dimension--include and rank matches from the same blog higher, even if according to my history, I haven't read that post.
jacquesm 17y ago
for your #2 see 'boomtango'
dkersten 17y ago
I was thinking about #2 last night actually. I was reading an issue of CACM and thought it would be useful to keep a database of all journals, research papers and magazines I read and tagging them with categories and other metadata. Then in a few years, I might think "hmm, I'm looking for X" and search the database and find a list of journals, research papers and magazines that are relevant. I think that it would work, since after a while I would learn which tags mean what, so later I'd think "XYZ, thats tags A, B and C - search!"
- jimmybot 17y ago
  Do you use Zotero? If so, how does it or does it not fit your needs?
  - dkersten 17y ago
    I'd never heard of it, actually. I will give it a look - thanks!
jokermatt999 17y ago
The best solution I've found for #2 is Firefox's Awesome Bar. However, that requires you remember at least one word in the title, at which point Google is probably about as effective.
ntoshev 17y ago
What about http://www.archive.org http://www.archive.org doesn't work for you about #1? It's centralized but this is hardly a practical problem; search is also centralized and much more important.
- aristus 17y ago
  The biggest problem is that a centralized archive is at the mercy of copyright holders ex-post-facto. I defy you to find more than a few recent articles from the NYTimes.com on archive.org. Things get taken down all the time.

by aristus 17y ago

jimmybot 17y ago

I've thought about number two as well. How about a combination of caching all pages you read and how long you are spending on each page (and maybe if you scrolled as well)? The idea is to save enough data that you can have an algorithm that will automatically differentiate between stuff you closed within a couple of seconds and didn't really read and stuff that interested you and you actually read through. And the reason for doing this automatically is it seems very difficult to have the discipline take an action every time you see a page that you may want to save so that you can search through again (such as clicking a box for every such page). Whatever mechanism it is, it needs to account for stuff read through readers and over multiple computers/devices too. I don't want to miss a search result just because I was reading it on an iPhone or in my Google Reader. Or instead of caching locally, a search interface that would accept a long list of urls to look through.

ntoshev 17y ago
I have thought about it as well. The signals that could determine relevancy in your browser history are likely different form the ones that determine it for web search, and measuring engagement with a page is a good example (it would help web search as well, but it would be harder to measure). Other things you'd take into account are: * continuity in time - reading several pages on similar topics one after the other. * analysis of the subgraph of the web graph that you visited * you'd be more aggressive with query expansion because you work with much smaller corpus, e.g. you'd want to take hypernyms and hyponyms into account
- jimmybot 17y ago
  Ah yes, engagement is the word. Interesting. I'd imagine the query expansion should also work across the author dimension--include and rank matches from the same blog higher, even if according to my history, I haven't read that post.

jacquesm 17y ago

for your #2 see 'boomtango'

dkersten 17y ago

I was thinking about #2 last night actually. I was reading an issue of CACM and thought it would be useful to keep a database of all journals, research papers and magazines I read and tagging them with categories and other metadata. Then in a few years, I might think "hmm, I'm looking for X" and search the database and find a list of journals, research papers and magazines that are relevant. I think that it would work, since after a while I would learn which tags mean what, so later I'd think "XYZ, thats tags A, B and C - search!"

jimmybot 17y ago
Do you use Zotero? If so, how does it or does it not fit your needs?
- dkersten 17y ago
  I'd never heard of it, actually. I will give it a look - thanks!

jokermatt999 17y ago

The best solution I've found for #2 is Firefox's Awesome Bar. However, that requires you remember at least one word in the title, at which point Google is probably about as effective.

ntoshev 17y ago

What about http://www.archive.org http://www.archive.org doesn't work for you about #1? It's centralized but this is hardly a practical problem; search is also centralized and much more important.

aristus 17y ago
The biggest problem is that a centralized archive is at the mercy of copyright holders ex-post-facto. I defy you to find more than a few recent articles from the NYTimes.com on archive.org. Things get taken down all the time.