3 ms·

I've always wondered why browsers don't put the contents of pages I visit into something like clucene, so I can search pages in my history by their contents, no

by mhaymo 10y ago

I've always wondered why browsers don't put the contents of pages I visit into something like clucene, so I can search pages in my history by their contents, not just their titles and metadata. I'm sure there are security and storage tradeoffs, but to me this seems like a much more complete solution than this revamped history UI.

yread 10y ago
Opera used to do that from 10.50 http://help.opera.com/Mac/10.50/en/history.html http://help.opera.com/Mac/10.50/en/history.html
dingaling 10y ago
There is a recoll extension for Firefox, which updates an index visible from the desktop program. Not quite as neat as integrated history searching in the browser but still useful. https://addons.mozilla.org/en-US/firefox/addon/recoll-indexer-1/ https://addons.mozilla.org/en-US/firefox/addon/recoll-indexe... The one complication I found was ensuring that I blacklisted sensitive websites such as banks.
waldir 10y ago
You might be interested in https://worldbrain.io https://worldbrain.io
NoGravitas 10y ago
Back around 2000, I wrote a web proxy server that did this. It was a proxy server mainly because browser extensions were not a thing back then. I never released it because I never solved some performance issues (it was using a very naive implementation in a lot of ways). I did get a lot of use out of it for about a year, though. I'm sure there are security/privacy trade-offs. Storage is not such a big thing. I found that index size leveled off somewhat after a point. It probably would use enough storage to make it unreasonable for mobile browsers.
- mistermann 10y ago
  Man, if this was a thing I would definitely run it. All the plugin approaches fall down for me as I use FF and Chrome interchangeably for different tasks. This would be a good additional feature for Pinboard perhaps (which I've been intending to subscribe to for ages but never get around to).
  - NoGravitas 10y ago
    Unfortunately, HTTP proxies are kind of a thing of the past. Back when I wrote it, it was easy to implement a proxy (HTTP 1.0 with some HTTP 1.1 features). Now there are two issues. The smaller one is that you have to handle a lot of complexity from HTTP 1.1. The bigger one is that increasing use of HTTPS (a good thing!) means that things like caching, ad-blocking, and history search absolutely have to be in the browser. The author of Polipo, one of the best caching proxies, has declared it obsolete for this reason. It would be a good feature for the deluxe Pinboard subscription. As long as you're mirroring the content, you might as well index it with Lucene or something.
kalleboo 10y ago
I thought Safari did that, but it seems it only does it for bookmarks - did that change at some point?

by mhaymo 10y ago

yread 10y ago

Opera used to do that from 10.50 http://help.opera.com/Mac/10.50/en/history.html http://help.opera.com/Mac/10.50/en/history.html

dingaling 10y ago

There is a recoll extension for Firefox, which updates an index visible from the desktop program. Not quite as neat as integrated history searching in the browser but still useful. https://addons.mozilla.org/en-US/firefox/addon/recoll-indexer-1/ https://addons.mozilla.org/en-US/firefox/addon/recoll-indexe... The one complication I found was ensuring that I blacklisted sensitive websites such as banks.

waldir 10y ago

You might be interested in https://worldbrain.io https://worldbrain.io

NoGravitas 10y ago

Back around 2000, I wrote a web proxy server that did this. It was a proxy server mainly because browser extensions were not a thing back then. I never released it because I never solved some performance issues (it was using a very naive implementation in a lot of ways). I did get a lot of use out of it for about a year, though. I'm sure there are security/privacy trade-offs. Storage is not such a big thing. I found that index size leveled off somewhat after a point. It probably would use enough storage to make it unreasonable for mobile browsers.

mistermann 10y ago
Man, if this was a thing I would definitely run it. All the plugin approaches fall down for me as I use FF and Chrome interchangeably for different tasks. This would be a good additional feature for Pinboard perhaps (which I've been intending to subscribe to for ages but never get around to).
- NoGravitas 10y ago
  Unfortunately, HTTP proxies are kind of a thing of the past. Back when I wrote it, it was easy to implement a proxy (HTTP 1.0 with some HTTP 1.1 features). Now there are two issues. The smaller one is that you have to handle a lot of complexity from HTTP 1.1. The bigger one is that increasing use of HTTPS (a good thing!) means that things like caching, ad-blocking, and history search absolutely have to be in the browser. The author of Polipo, one of the best caching proxies, has declared it obsolete for this reason. It would be a good feature for the deluxe Pinboard subscription. As long as you're mirroring the content, you might as well index it with Lucene or something.

kalleboo 10y ago

I thought Safari did that, but it seems it only does it for bookmarks - did that change at some point?