10 ms·

Show HN: Never lose a website again

78 points by flippyhead 12y ago

saint-loup 12y ago
I may be blind, but you don't seem to inform the user that the addon is currently Chrome-only before he has suscribed. It's misleading.
PeterWhittaker 12y ago
Why do so many sites require cookies just to tell me what they are all about? Yes, I am of the lunatic fringe that blocks all cookies by default... ...and by default more and more web sites lose me as a potential client/user/whatev because they require cookies just to display static welcoming information... ...I lose nothing by this, as far as I can tell. Perhaps ignorance really is bliss. :->
- dguaraglia 12y ago
  I guess most websites nowadays are built using one of the myriad web frameworks out there (Django, RoR, you name it.) Most of this frameworks enable sessions by default, simply because it's what most websites will want if they manage any kind of state. Nothing nefarious about it.
  - jarrett 12y ago
    > Most of this frameworks enable sessions by default True, but in my experience, the major frameworks don't automatically lock out users with cookies disabled. For example, on a Rails app with no before_filter on the homepage, you can start the server and do this: echo "GET / HTTP/1.1" | nc localhost 3000 You should get back the homepage HTML.
- diafygi 12y ago
  One legit reason is to set a CSRF cookie. If there's a login or signup form on the page, many frameworks add a CSRF field to the form that should match the cookie. When you POST the form, the cookie and the field must match.
  - PeterWhittaker 12y ago
    Sure, that's legit - if login or signup is required. But for pages containing information intended to encourage a user to spend time on a site, to get convinced they need to sign-up, it's counter-productive.
- flippyhead 12y ago
  This is a great point and not one I'd ever even considered. You are right, the page you see first -- before signing up -- really doesn't need much. I am, in fact, using the Meteor stack for the entire thing, even the informational pages. But that was mostly a convenience.
  - PeterWhittaker 12y ago
    Thanks for answering and for accepting this comment - I needed the validation, since I lost two karma points on the comment. Ironic, no? :->
someone5555 12y ago
To be blunt, this project will never gain traction because not enough people are willing to store their browsing history somewhere outside of their control. The back button issue shows a lack of attention to detail. Something that's extremely important when dealing with sensitive personal data.
- flippyhead 12y ago
  Please know this is the first exposure this project has ever had. I really did try, in my spare time, to get this thing perfect before soliciting feedback from this wondrous community but -- alas! -- there be bugs. It's this kind of feedback that I was seeking and I intend to incorporate as I drive towards a broader release. Part of what I'm trying to validate is exactly the point you raised: will people generally be too freaked out to store their browsing history "in the cloud"? The next thing I'm hoping to determine is if it'll be enough to encrypt this content is such a way that people are OK with it being stored externally or if only a locally installed version will do. Security aside, there's a great deal of advantage to offer this service hosted. Thanks for your feedback!
  - lolatu54 12y ago
    I think the only way people would be comfortable is if you use strong one-way encryption, meaning you yourself cannot decrypt a user's data. But to enable trust, and being a startup, you would have to have your code reviewed by a reliable third party or open-sourced for public review. Google gets away with it because they have built a brand with a reputation that some people trust enough with their data (I don't personally, but enough do). A much easier path is to create a version that works locally with no external communication. Both options would be ideal, but perhaps more work than you care to take on. No matter how it pans out it will be a great learning experience.
minusSeven 12y ago
not available on Firefox. Can't use it.
aeflash 12y ago
Ah, so it's like Firefox's URL bar, except with full-text search, rather than just the URL/Title.
mooism2 12y ago
What browsers does it work with? Where's your privacy policy?
- bradbeattie 12y ago
  Moreover, how does this work with sites that make use of ajax content? It'd be frustrating to assume that it's recording just fine, only to later find that patches of your history couldn't be recorded.
  - flippyhead 12y ago
    That's a great point. At the moment it only records what's in the DOM (stripped of all HTML) after the first page load event. I'm working on how to include AJAX content but it's not nearly as straight forward.
    - waterside81 12y ago
      Take a look at readability.js (https://github.com/Kerrick/readability-js https://github.com/Kerrick/readability-js) and extract/upload the main DOM content after all the JS trickery has completed.
      - bradbeattie 12y ago
        Assuming it completes. I can easily imagine a page with a news ticker that updates once every few seconds.
jgh 12y ago
Your site breaks the back button (Chrome 35 OS X Mavericks)
- flippyhead 12y ago
  You are right! I'm working on a fix.
  - brk 12y ago
    Out of curiosity, what are you doing with such a simple site (and I don't mean that in a bad way) that requires any conscious effort to fix? It seems like you really need to go out of your way to fuck up the back button and for a fix to be "work". But, recent nuanced trends in web design/navigation aren't my top skill, so I'm asking the question honestly.
    - zevyoura 12y ago
      The site is built with the meteor framework, so it's not just the simple implementation you might expect from the layout. I imagine the features of meteor are useful in the actual app itself, which is probably not a simple site.
      - rurounijones 12y ago
        Basically, wrong tool for the job. The top page should probably be static, no need to involve meteor at all. The rest? Maybe meteor is the best choice, cannot really say.
    - flippyhead 12y ago
      Honestly, I've been taken a bit by surprise that the back button bug has been such a big issue. I did some testing among friends (with a group of five or so startup, programmer types) and, believe it or not, it never came up.. That said, it'll be fixed in short order. To the other comments, the service is considerably more complex (and yes, built on Meteor, MongoDB and ElasticSearch) as it serves up search results and updates in real time. Though the UI is simple, it does take a fair bit of effort to organize all the browsing information and index such that search results are relevant.
mappum 12y ago
I don't understand, what is the advantage over using Chrome's history menu?
- flippyhead 12y ago
  Chrome doesn't index the contents of the page and is limited in how much history it records. This does full text indexing of the pages you visit and lasts forever. It's backed by elasticsearch.
pbhjpbhj 12y ago
Do you just record n-grams indexed against the page url, are you then uploading that index? If you're not uploading it how is there no "local version" available? It's an interesting idea. Personally I have a script that wgets all the pages I bookmark and I very rarely use that content. What use cases are you anticipating?
- gwern 12y ago
  What does indexing mean here? I'm not sure what it means to 'index' but not 'store' webpages as the frontpage says.
  - pbhjpbhj 12y ago
    I assumed they meant that they extracted meta-data about the page (page mentions "Winnie the Pooh" for example, like a book index) and kept that associated with a key (eg URL) but didn't actually store the content per se. I was hoping the answer to my question would include that sort of detail - what's stored and where.
    - flippyhead 12y ago
      That's exactly right. The indexed content is stored (as an inserted index) but not the plain text.
- boyter 12y ago
  It would be useful for pulling back pages that have fallen off the internet and are not in the wayback machine. Thats a use case I have hit a few times. I even started backing up useful posts just in case they died. http://gigablast.com/rants.html http://gigablast.com/rants.html is an example. Its a really good insight into the creation of a search engine which really should be preserved. One that did dissapear but has since come back is this post http://widgetsandshit.com/teddziuba/2010/10/taco-bell-programming.html http://widgetsandshit.com/teddziuba/2010/10/taco-bell-progra... which I went looking for a few years ago but had dissapeared from all search indexes.
ladybro 12y ago
Ha, very random but I remember you from an Airbnb listing a few months ago. Almost lived with you in Seattle for a couple weeks! Best of luck.
drdaeman 12y ago
I was just looking for such indexer this exact evening. Unfortunately, logon via Twitter fails with 500 error bar flashing at the site's top and logon via FB fails with "App Not Setup: The developers of this app have not set up this app properly for Facebook Login.", so I can't try it. Nonetheless, my biggest worry is why this is a service instead of a standalone package. (Actually, I've considered trying with the hope plugin may be possibly FOSS and if so researching whenever it could be hacked to be used with locally-installed Solr/Lucene server) I'm not really comfortable with directly or indirectly sharing my browsing history with most third parties.
- flippyhead 12y ago
  Bah, I'm sorry! The Facebook login has now been fixed. I totally hear you on some people not wanting to share browsing history externally. This first version was easiest to do as a hosted service. Next up I intend to package it up as an installable app.
  - volent 12y ago
    Twitter login still not working
- flippyhead 12y ago
  That said, I have found it handy to have access to this searchable history for different locations and devices. For example on my phone.
- Jonovono 12y ago
  Local version: https://github.com/idibidiart/AllSeeingEye https://github.com/idibidiart/AllSeeingEye
- purplerails 12y ago
  Please also look at https://www.purplerails.com/ https://www.purplerails.com/ I tried address the privacy angle from day 1. Data is encrypted and only ciphertext is sent to the cloud. Index is stored only on your own machine. Searching occurs on your own machine. A couple of cool features are that it also saves an exact HTML copy of the page including images and stuff if you read a page "long enough" (currently 90 seconds). Been in beta for a while. Thanks for feedback.
ark15 12y ago
Two issues with your landing page 1. major - Breaks back button 2. minor(nitpick) - If I click on the green "notify me" button and then "cancel" on the prompt dialog instead of "OK", it still shows me the green "Success - you will be notified" message Good luck though!
- mmaunder 12y ago
  "Never lose a website again" ...because you will never leave this one.
  - flippyhead 12y ago
    Ha!
meritt 12y ago
I'm a bit confused. Is this essentially a browser plugin which extracts plaintext of your browsing and then sends it off to fetching.io's servers for indexing, and presumably some sort of search box area to let me search those?
sgibat 12y ago
I love this idea. Chrome's history is insufficient; I swear, it often doesn't record sites I've visited. But.. I'm not ready to trust an unknown process sending my entire browsing history to unknown servers. Excited for a potential local version.
- tgb 12y ago
  Agreed. I learned recently that you get better history search by going to Google and having it only show 'visited sites'.
  - mixologic 12y ago
    Visited pages only seems to include the sites you visit immediately after doing a google search. It does not include pages found through normal browsing.
- ryanpardieck 12y ago
  Yeah, I noticed just today that it doesn't record everything! Very strange. Could turbolinks-alike implementations be throwing it off?
- joshvm 12y ago
  Chrome's history is inordinately awful. Get this addon: https://chrome.google.com/webstore/detail/better-history/obciceimmggglbmelaidpjlmodcebijb?hl=en https://chrome.google.com/webstore/detail/better-history/obc...
  - joshschreuder 12y ago
    I've always thought this, so thanks for the extension, looks pretty good. It would be nice to have some geek stats like a top x list of your most visited sites.
bujatt 12y ago
I like the concept of this service. Basically we should be able to find _any_ piece of information that we have already encountered with anytime in our lifetime. This service is indexing this layer of information. I would add another layer for information with which we engage more: e.g. liking, linking, sharing etc.
soundoflight 12y ago
As an Opera user, there has been many times where the in-page content searching has helped me find stuff. When I'm in different browsers, that loses its usefulness. I'll definitely come back to this when the local version is done.
vertak 12y ago
Is each user's index stored somewhere in their browser or on your servers? It would probably ease peoples' minds if they knew their history was stored locally, but I can see the performance hit that might make.
eglover 12y ago
If this doesn't repeat websites and categorizes them in a neat manner it would be amazing. My obsessive bookmarking led me to throw this up: http://goo.gl/WNw5OG http://goo.gl/WNw5OG I look forward to seeing what you come up with.
KhalilK 12y ago
hmmm, privacy?
arikrak 12y ago
Looks interesting, I've often wanted to find old pages I've visited. Though I'll wait until it has a track record, before I trust it with my history. It would also be important to rank searches well, since otherwise an entire history may have too much. Though this will be a hard area to take on Google...
ibrad 12y ago
I am going to wait for the local version for this one. I am working on a project that needs the full browsing history including content. You might have read read it here on HN[1] The goal is to make my computer a search engine that can also recommend articles based on the ones I have visited. It can also check websites to see if there is any new content. The project is still in its very early stages and any tool I can use will be very helpful. This looks just like what I need. [1]: https://news.ycombinator.com/item?id=7822859 https://news.ycombinator.com/item?id=7822859
Jonovono 12y ago
For people that want something like this but locally: https://github.com/idibidiart/AllSeeingEye https://github.com/idibidiart/AllSeeingEye
- hupili 12y ago
  nice pointer! Not sure what amount it can support. It stores screenshots for the pages, which is a lot consumption of space.
smoe 12y ago
Why server side indexing and not use something like lunarjs[1] from the beginning? Like the Idea of fulltext search on history, but I'm not going to send my full history to some random dude on the internet. No offence intended:) [1] ”Simple full-text search in your browser” http://lunrjs.com/ http://lunrjs.com/
- eli 12y ago
  I always thought it surprisingly how poorly the "Search History" feature works in Chrome. Could be super useful if it worked.
  - Honzo 12y ago
    The idea is simple enough. My concerns would be how many resources it would use after a while of use. It's basically duplicating your already existing history.
    - woah 12y ago
      Maybe not, I think you can get the history from chrome and perhaps just have the app store an index.
- flippyhead 12y ago
  None taken! I looked at lunarjs (and a few other options) but felt that the more sophisticated features of ElasticSearch were worth it (at least to me). I'm curious, would you be more comfortable if all your content were encrypted such that not even the app developer (me) could read it? Or would only a local index do?
  - ojilles 12y ago
    No, not really
  - devcpp 12y ago
    Most people would be okay with plaintext content sitting on your server and even publicly accessible by anyone. But this is HN, and we wouldn't use even an AES-512-encrypted service if it has to leave our computers. Our password could still be cracked with enough computing power. So, we'd all be extremely happy if you could make a version for paranoid people like us. Also, I don't know how your indexing works, but please make it easy to backup (just allowing us to define the index folder would do). Possible data loss is the tradeoff with local content.
xanderstrike 12y ago
Why do I need to create an account for what sounds like a browser extension?
serf 12y ago
when is firefox support planned?
PeterGriffin 12y ago
Interesting service, but if you consider the market, it's the cross-section of people... - who are both anal-retentive enough to want to index the content of every page they visit, and yet... - are not anal-retentive enough to want to index the content of any page visited on a mobile device, and also... - don't care about their privacy by sending the address and index of every single page they ever visit to someone. Good luck.
purplerails 12y ago
Shameless plug: https://www.purplerails.com/ https://www.purplerails.com/ (1) Saves an exact copy of the page also. (2) Indexes the text. (3) Encrypted (search occurs on your computer). Been in beta for a while. Thanks for feedback.
- andyhmltn 12y ago
  Love the cartoon on your front page!
  - purplerails 12y ago
    Thank you for your kind words!
tempestn 12y ago
Cool. I've definitely had a desire for something like this in the past. It's mostly been filled by Evernote and its web clipper now. Any time I have a vague inkling that I might want a page again sometime, I clip it, so I can easily find it with a future search. (Often by accident since you can configure the clipper to show Evernote notes alongside search engine results.) The downside compared to something like this is that it only works if I have the foresight to clip a page. But the upside is that I don't end up indexing all the crap I definitely won't want again, so it's easier to find things in the remainder.
- flippyhead 12y ago
  Thanks. My hope is that the search is good enough that it doesn't matter how much is indexed -- you'll always be able to find what you are looking for. This is one reason, to start, I felt it was easier to index content server side..
  - tempestn 12y ago
    Of course, there's no reason not to do both. A service like yours could catch anything that falls through the cracks with EN. If you could eventually create browser extensions like theirs that add results alongside search engine results, it might would help people rediscover pages they've used in the past.
Flenser 12y ago
What does it look like? Would be nice to have a demo page with an example index to try without having to sign up. I bet you could A/B test it to see what affected sign-up rates too.

78 points by flippyhead 12y ago

I may be blind, but you don't seem to inform the user that the addon is currently Chrome-only before he has suscribed. It's misleading.

PeterWhittaker 12y ago

Why do so many sites require cookies just to tell me what they are all about? Yes, I am of the lunatic fringe that blocks all cookies by default... ...and by default more and more web sites lose me as a potential client/user/whatev because they require cookies just to display static welcoming information... ...I lose nothing by this, as far as I can tell. Perhaps ignorance really is bliss. :->

dguaraglia 12y ago
I guess most websites nowadays are built using one of the myriad web frameworks out there (Django, RoR, you name it.) Most of this frameworks enable sessions by default, simply because it's what most websites will want if they manage any kind of state. Nothing nefarious about it.
- jarrett 12y ago
  > Most of this frameworks enable sessions by default True, but in my experience, the major frameworks don't automatically lock out users with cookies disabled. For example, on a Rails app with no before_filter on the homepage, you can start the server and do this: echo "GET / HTTP/1.1" | nc localhost 3000 You should get back the homepage HTML.
diafygi 12y ago
One legit reason is to set a CSRF cookie. If there's a login or signup form on the page, many frameworks add a CSRF field to the form that should match the cookie. When you POST the form, the cookie and the field must match.
- PeterWhittaker 12y ago
  Sure, that's legit - if login or signup is required. But for pages containing information intended to encourage a user to spend time on a site, to get convinced they need to sign-up, it's counter-productive.
flippyhead 12y ago
This is a great point and not one I'd ever even considered. You are right, the page you see first -- before signing up -- really doesn't need much. I am, in fact, using the Meteor stack for the entire thing, even the informational pages. But that was mostly a convenience.
- PeterWhittaker 12y ago
  Thanks for answering and for accepting this comment - I needed the validation, since I lost two karma points on the comment. Ironic, no? :->

someone5555 12y ago

To be blunt, this project will never gain traction because not enough people are willing to store their browsing history somewhere outside of their control. The back button issue shows a lack of attention to detail. Something that's extremely important when dealing with sensitive personal data.

flippyhead 12y ago
Please know this is the first exposure this project has ever had. I really did try, in my spare time, to get this thing perfect before soliciting feedback from this wondrous community but -- alas! -- there be bugs. It's this kind of feedback that I was seeking and I intend to incorporate as I drive towards a broader release. Part of what I'm trying to validate is exactly the point you raised: will people generally be too freaked out to store their browsing history "in the cloud"? The next thing I'm hoping to determine is if it'll be enough to encrypt this content is such a way that people are OK with it being stored externally or if only a locally installed version will do. Security aside, there's a great deal of advantage to offer this service hosted. Thanks for your feedback!
- lolatu54 12y ago
  I think the only way people would be comfortable is if you use strong one-way encryption, meaning you yourself cannot decrypt a user's data. But to enable trust, and being a startup, you would have to have your code reviewed by a reliable third party or open-sourced for public review. Google gets away with it because they have built a brand with a reputation that some people trust enough with their data (I don't personally, but enough do). A much easier path is to create a version that works locally with no external communication. Both options would be ideal, but perhaps more work than you care to take on. No matter how it pans out it will be a great learning experience.

minusSeven 12y ago

not available on Firefox. Can't use it.

aeflash 12y ago

Ah, so it's like Firefox's URL bar, except with full-text search, rather than just the URL/Title.

mooism2 12y ago

What browsers does it work with? Where's your privacy policy?

bradbeattie 12y ago
Moreover, how does this work with sites that make use of ajax content? It'd be frustrating to assume that it's recording just fine, only to later find that patches of your history couldn't be recorded.
- flippyhead 12y ago
  That's a great point. At the moment it only records what's in the DOM (stripped of all HTML) after the first page load event. I'm working on how to include AJAX content but it's not nearly as straight forward.
  - waterside81 12y ago
    Take a look at readability.js (https://github.com/Kerrick/readability-js https://github.com/Kerrick/readability-js) and extract/upload the main DOM content after all the JS trickery has completed.
    - bradbeattie 12y ago
      Assuming it completes. I can easily imagine a page with a news ticker that updates once every few seconds.

jgh 12y ago

Your site breaks the back button (Chrome 35 OS X Mavericks)

flippyhead 12y ago
You are right! I'm working on a fix.
- brk 12y ago
  Out of curiosity, what are you doing with such a simple site (and I don't mean that in a bad way) that requires any conscious effort to fix? It seems like you really need to go out of your way to fuck up the back button and for a fix to be "work". But, recent nuanced trends in web design/navigation aren't my top skill, so I'm asking the question honestly.
  - zevyoura 12y ago
    The site is built with the meteor framework, so it's not just the simple implementation you might expect from the layout. I imagine the features of meteor are useful in the actual app itself, which is probably not a simple site.
    - rurounijones 12y ago
      Basically, wrong tool for the job. The top page should probably be static, no need to involve meteor at all. The rest? Maybe meteor is the best choice, cannot really say.
  - flippyhead 12y ago
    Honestly, I've been taken a bit by surprise that the back button bug has been such a big issue. I did some testing among friends (with a group of five or so startup, programmer types) and, believe it or not, it never came up.. That said, it'll be fixed in short order. To the other comments, the service is considerably more complex (and yes, built on Meteor, MongoDB and ElasticSearch) as it serves up search results and updates in real time. Though the UI is simple, it does take a fair bit of effort to organize all the browsing information and index such that search results are relevant.

mappum 12y ago

I don't understand, what is the advantage over using Chrome's history menu?

flippyhead 12y ago
Chrome doesn't index the contents of the page and is limited in how much history it records. This does full text indexing of the pages you visit and lasts forever. It's backed by elasticsearch.

pbhjpbhj 12y ago

Do you just record n-grams indexed against the page url, are you then uploading that index? If you're not uploading it how is there no "local version" available? It's an interesting idea. Personally I have a script that wgets all the pages I bookmark and I very rarely use that content. What use cases are you anticipating?

gwern 12y ago
What does indexing mean here? I'm not sure what it means to 'index' but not 'store' webpages as the frontpage says.
- pbhjpbhj 12y ago
  I assumed they meant that they extracted meta-data about the page (page mentions "Winnie the Pooh" for example, like a book index) and kept that associated with a key (eg URL) but didn't actually store the content per se. I was hoping the answer to my question would include that sort of detail - what's stored and where.
  - flippyhead 12y ago
    That's exactly right. The indexed content is stored (as an inserted index) but not the plain text.
boyter 12y ago
It would be useful for pulling back pages that have fallen off the internet and are not in the wayback machine. Thats a use case I have hit a few times. I even started backing up useful posts just in case they died. http://gigablast.com/rants.html http://gigablast.com/rants.html is an example. Its a really good insight into the creation of a search engine which really should be preserved. One that did dissapear but has since come back is this post http://widgetsandshit.com/teddziuba/2010/10/taco-bell-programming.html http://widgetsandshit.com/teddziuba/2010/10/taco-bell-progra... which I went looking for a few years ago but had dissapeared from all search indexes.

ladybro 12y ago

Ha, very random but I remember you from an Airbnb listing a few months ago. Almost lived with you in Seattle for a couple weeks! Best of luck.

drdaeman 12y ago

I was just looking for such indexer this exact evening. Unfortunately, logon via Twitter fails with 500 error bar flashing at the site's top and logon via FB fails with "App Not Setup: The developers of this app have not set up this app properly for Facebook Login.", so I can't try it. Nonetheless, my biggest worry is why this is a service instead of a standalone package. (Actually, I've considered trying with the hope plugin may be possibly FOSS and if so researching whenever it could be hacked to be used with locally-installed Solr/Lucene server) I'm not really comfortable with directly or indirectly sharing my browsing history with most third parties.

flippyhead 12y ago
Bah, I'm sorry! The Facebook login has now been fixed. I totally hear you on some people not wanting to share browsing history externally. This first version was easiest to do as a hosted service. Next up I intend to package it up as an installable app.
- volent 12y ago
  Twitter login still not working
flippyhead 12y ago
That said, I have found it handy to have access to this searchable history for different locations and devices. For example on my phone.
Jonovono 12y ago
Local version: https://github.com/idibidiart/AllSeeingEye https://github.com/idibidiart/AllSeeingEye
purplerails 12y ago
Please also look at https://www.purplerails.com/ https://www.purplerails.com/ I tried address the privacy angle from day 1. Data is encrypted and only ciphertext is sent to the cloud. Index is stored only on your own machine. Searching occurs on your own machine. A couple of cool features are that it also saves an exact HTML copy of the page including images and stuff if you read a page "long enough" (currently 90 seconds). Been in beta for a while. Thanks for feedback.

ark15 12y ago

Two issues with your landing page 1. major - Breaks back button 2. minor(nitpick) - If I click on the green "notify me" button and then "cancel" on the prompt dialog instead of "OK", it still shows me the green "Success - you will be notified" message Good luck though!

mmaunder 12y ago
"Never lose a website again" ...because you will never leave this one.
- flippyhead 12y ago
  Ha!

meritt 12y ago

I'm a bit confused. Is this essentially a browser plugin which extracts plaintext of your browsing and then sends it off to fetching.io's servers for indexing, and presumably some sort of search box area to let me search those?

sgibat 12y ago

I love this idea. Chrome's history is insufficient; I swear, it often doesn't record sites I've visited. But.. I'm not ready to trust an unknown process sending my entire browsing history to unknown servers. Excited for a potential local version.

tgb 12y ago
Agreed. I learned recently that you get better history search by going to Google and having it only show 'visited sites'.
- mixologic 12y ago
  Visited pages only seems to include the sites you visit immediately after doing a google search. It does not include pages found through normal browsing.
ryanpardieck 12y ago
Yeah, I noticed just today that it doesn't record everything! Very strange. Could turbolinks-alike implementations be throwing it off?
joshvm 12y ago
Chrome's history is inordinately awful. Get this addon: https://chrome.google.com/webstore/detail/better-history/obciceimmggglbmelaidpjlmodcebijb?hl=en https://chrome.google.com/webstore/detail/better-history/obc...
- joshschreuder 12y ago
  I've always thought this, so thanks for the extension, looks pretty good. It would be nice to have some geek stats like a top x list of your most visited sites.

bujatt 12y ago

I like the concept of this service. Basically we should be able to find _any_ piece of information that we have already encountered with anytime in our lifetime. This service is indexing this layer of information. I would add another layer for information with which we engage more: e.g. liking, linking, sharing etc.

soundoflight 12y ago

As an Opera user, there has been many times where the in-page content searching has helped me find stuff. When I'm in different browsers, that loses its usefulness. I'll definitely come back to this when the local version is done.

vertak 12y ago

Is each user's index stored somewhere in their browser or on your servers? It would probably ease peoples' minds if they knew their history was stored locally, but I can see the performance hit that might make.

eglover 12y ago

If this doesn't repeat websites and categorizes them in a neat manner it would be amazing. My obsessive bookmarking led me to throw this up: http://goo.gl/WNw5OG http://goo.gl/WNw5OG I look forward to seeing what you come up with.

KhalilK 12y ago

hmmm, privacy?

arikrak 12y ago

Looks interesting, I've often wanted to find old pages I've visited. Though I'll wait until it has a track record, before I trust it with my history. It would also be important to rank searches well, since otherwise an entire history may have too much. Though this will be a hard area to take on Google...

ibrad 12y ago

I am going to wait for the local version for this one. I am working on a project that needs the full browsing history including content. You might have read read it here on HN[1] The goal is to make my computer a search engine that can also recommend articles based on the ones I have visited. It can also check websites to see if there is any new content. The project is still in its very early stages and any tool I can use will be very helpful. This looks just like what I need. [1]: https://news.ycombinator.com/item?id=7822859 https://news.ycombinator.com/item?id=7822859

Jonovono 12y ago

For people that want something like this but locally: https://github.com/idibidiart/AllSeeingEye https://github.com/idibidiart/AllSeeingEye

hupili 12y ago
nice pointer! Not sure what amount it can support. It stores screenshots for the pages, which is a lot consumption of space.

smoe 12y ago

Why server side indexing and not use something like lunarjs[1] from the beginning? Like the Idea of fulltext search on history, but I'm not going to send my full history to some random dude on the internet. No offence intended:) [1] ”Simple full-text search in your browser” http://lunrjs.com/ http://lunrjs.com/

eli 12y ago
I always thought it surprisingly how poorly the "Search History" feature works in Chrome. Could be super useful if it worked.
- Honzo 12y ago
  The idea is simple enough. My concerns would be how many resources it would use after a while of use. It's basically duplicating your already existing history.
  - woah 12y ago
    Maybe not, I think you can get the history from chrome and perhaps just have the app store an index.
flippyhead 12y ago
None taken! I looked at lunarjs (and a few other options) but felt that the more sophisticated features of ElasticSearch were worth it (at least to me). I'm curious, would you be more comfortable if all your content were encrypted such that not even the app developer (me) could read it? Or would only a local index do?
- ojilles 12y ago
  No, not really
- devcpp 12y ago
  Most people would be okay with plaintext content sitting on your server and even publicly accessible by anyone. But this is HN, and we wouldn't use even an AES-512-encrypted service if it has to leave our computers. Our password could still be cracked with enough computing power. So, we'd all be extremely happy if you could make a version for paranoid people like us. Also, I don't know how your indexing works, but please make it easy to backup (just allowing us to define the index folder would do). Possible data loss is the tradeoff with local content.

xanderstrike 12y ago

Why do I need to create an account for what sounds like a browser extension?

serf 12y ago

when is firefox support planned?

PeterGriffin 12y ago

Interesting service, but if you consider the market, it's the cross-section of people... - who are both anal-retentive enough to want to index the content of every page they visit, and yet... - are not anal-retentive enough to want to index the content of any page visited on a mobile device, and also... - don't care about their privacy by sending the address and index of every single page they ever visit to someone. Good luck.

purplerails 12y ago

Shameless plug: https://www.purplerails.com/ https://www.purplerails.com/ (1) Saves an exact copy of the page also. (2) Indexes the text. (3) Encrypted (search occurs on your computer). Been in beta for a while. Thanks for feedback.

andyhmltn 12y ago
Love the cartoon on your front page!
- purplerails 12y ago
  Thank you for your kind words!

tempestn 12y ago

Cool. I've definitely had a desire for something like this in the past. It's mostly been filled by Evernote and its web clipper now. Any time I have a vague inkling that I might want a page again sometime, I clip it, so I can easily find it with a future search. (Often by accident since you can configure the clipper to show Evernote notes alongside search engine results.) The downside compared to something like this is that it only works if I have the foresight to clip a page. But the upside is that I don't end up indexing all the crap I definitely won't want again, so it's easier to find things in the remainder.

flippyhead 12y ago
Thanks. My hope is that the search is good enough that it doesn't matter how much is indexed -- you'll always be able to find what you are looking for. This is one reason, to start, I felt it was easier to index content server side..
- tempestn 12y ago
  Of course, there's no reason not to do both. A service like yours could catch anything that falls through the cracks with EN. If you could eventually create browser extensions like theirs that add results alongside search engine results, it might would help people rediscover pages they've used in the past.

Flenser 12y ago

What does it look like? Would be nice to have a demo page with an example index to try without having to sign up. I bet you could A/B test it to see what affected sign-up rates too.