8 ms·
Show HN: WorldBrain – full text, local search of your browsing history
- DyslexicAtheist 8y agothis project has been around for a while, see also some interesting comments: https://news.ycombinator.com/item?id=13427360 https://news.ycombinator.com/item?id=13427360
- agotterer 8y agoI’ve tried to find a solution to bookmark searching for a while. I’ve never found a product I liked or trust. Lately I’ve been manually adding bookmarks to a custom google search engine. I’m considering building an extension that will add them directly or sync chrome bookmarks. I figure google already knows what I’ve searched, so I feel much less sketchy about it. Would this extension be interesting to anyone? It would be very simple, open source, and have no middle man. It would send links directly to a google CSE via their API.
- JohnStrangeII 8y agoI would have privacy concerns, because I'm not using Google all the time for my searches and there is no need to give them even more data. The only solution I'd accept is one where all data is stored and indexed locally and there are good guarantees that deleted entries are actually deleted and/or wiped from the indices.
- agotterer 8y agoI’m not suggesting that you have to use google for the originating search. But you would need to use CSE to index any content you wanted to search for later. Arguably you could do this with a new google account if you’re concerned.
- ChristianBundy 8y agoA few years ago I would have been excited about this, but I personally won't be giving Google any more data unless I'm forced to. I'd love a self-hosted and local-first option, but either way I wish you good luck on your potential project.
- o_____________o 8y agoThis product appears to index your bookmarks, did you see that?
- agotterer 8y agoYup. I’ve read about the product but haven’t tried it. I’ll likely check it out. My issue with these services is (and maybe this one is different) is that I have to run 3rd party software and extensions and I never know how long these companies/products will stay around.
- RealWorldBrain 8y agoOli here from the team building Memex. Yeah as mentioned the tool allows you to do bookmark full-text search. It's open-source, so it will stay no matter what. We build for resilience and for us as the WorldBrain.io company not needing to stick around in order for the service to survive. We see ourselves as the stewards of this tool, not the sole benefactor or proprietor. Hope it will be helpful to you :)
- diggan 8y agoOne of the reasons I keep coming back to Firefox is that Firefox seems to already be doing this. At least it searches all parts of the URL and the title, so I'm 99% successful at getting the right URL in the awesomebar when I'm looking for something. Chrome is just horrible when it comes to this, and I can never get back to previous pages when searching via the addressbar.
- DennisP 8y agoIn Chrome if you open the history and hit "show full history," you get a search bar that works better for history.
- exikyut 8y agoWhere's "show full history"? I don't see anything resembling that, nor any obscure dropdowns or popouts that might be hiding it.
- bobviolier 8y ago⌘+Y On the mac. It's the bottom option in the history menu.
- 4684499 8y agoThe "show full history" menu option in History menu they mentioned is Mac only. On Windows, you can see the "show full history" option by holding the Go Back button at top left (hover your mouse on it you'll see a tooltip suggesting this). The sad part is, that option just opens the built-in History page, just like pressing Ctrl + H, which I believe that's not what we wanted at all. I think Chrome just redefined "full" while it regularly deletes history entries older than three months.
- exikyut 8y agoAh, right, Linux has "show full history" in the Back button menu (which can also be opened by right-clicking). And yeah, ^H isn't fun(/useful).
- JetSpiegel 8y agoThis seems a very interesting project. I'll see how this works in practice. It starts to get icky when you notice the devs have a business plan to outsource the indexing to the cloud, but there's a commitment to keep the servers parts open source too, so that you can self host.
- RealWorldBrain 8y agoOli here from the team building Memex. Yeah indeed, having stuff in the cloud is not ideal when it comes to privacy and centralisation. One of our core values is privacy and data ownership. So we do our best to make our business not dependent on (analysing and selling) your data, and instead provide you with service value you're willing to pay for. We are built with interoperability in mind, that will allow you to switch providers of Memex and Memex Cloud without frictions, in case there are breaches of trust, or simply better service. We follow these values by currently building for offline first usage, where your data is locally indexed and searchable primarily. With our search technology you'll be able to get up to 5 years of your research done in the browser. For the cloud part, it is unfortunately not yet possible to do performant search on encrypted data, otherwise it would not be such a big issue to have your index in the cloud. Equally unfortunate is that it comes with a lot of drawbacks to replicate all your data on all nodes, as opposed to have a central point to query. Especially when we are looking at phone usages. There it is really not practical, so there is a need to have some sort of cloud - UX is still very important. Most people can't be bothered with the drawbacks of decentralised and distributed systems (yet). We hope to get that switch in multiple smaller steps that guide (non-technical) users through a smooth transition to a Memex system that is as distributed as possible. (Check out Dat https://datproject.org/ https://datproject.org/, a technology we likely use to make that first step possible) And as you already noted, this stuff will be self-hostable. We see ourselves as a service provider first and want to serve people who can't/don't want to run their own server. A bit like the Wordpress model. You can read more about our approaches to running this business in our vision post: worldbrain.io/vision
- JetSpiegel 8y agoThanks for the long response. I was wondering about mobile usage too, and agree that having a server somewhere available for queries is the best solution. Since this deals with such private data, having a possibility for self-hosting is the correct solution.
- mark_l_watson 8y agoI nice idea, but an open source browser plugin with all local storage would fit my needs better. I prototyped something roughly like this several years ago. I wrote a simple Firefox plugin that communicated with a locally running server written in Closure with a Clojurescript web app for browsing that used the same server backend. I stopped working on the because services like Evernote do a better job, at the loss of some privacy. Edit: I didn’t intend to imply that Evernote reads or uses user data.
- olejorgenb 8y agohttps://github.com/hedning/recoll-web https://github.com/hedning/recoll-web
- thatcat 8y agoThe description is horrible on their github page, do you know if this integrates your browser history into recoll or what exactly?
- o_____________o 8y ago> I nice idea, but an open source browser plugin with all local storage would fit my needs better. This seems like it's local search. Are you saying you wish it was open source? The only cloud feature seems to be something about highlighting.
- RealWorldBrain 8y agoGood news :) This tool is open-source, and runs fully locally (except, as pointed out the feature that enables you to share quotes, because for that stuff you unfortunately need a server, still) We custom built a search technology on IndexedDB and Dexie.js, which is capable of indexing around 5 years of your personal web-research locally in the browser.
- agussell 8y agoSadly there is a lot to improve to be useful. I had horrible results with the searches.
- mickelsen 8y agoTo think that the classic Opera browser used to do this within the search bar... man, good times those.
- warent 8y agoNice, now you just need to enable a feature to search anyone else's browsing history
- RealWorldBrain 8y agoOli here from the team building Memex :) Good news: that's the plan! worldbrain.io/vision_deck
- NeedMoreTea 8y agoSounds like the very first version of Google Desktop Search before they turned it into a widget engine.
- Pistos2 8y agoI get the point of this, but me personally, I prefer to have aspects of me forgotten/gone rather than remembered, stored and searchable in the future. Yes, it's true that, about once every two months, I am looking for something that I swear I came across on the Internet at some point. However, the rest of the time, I'm able to re-find it just by doing another search, whether on search-engine-of-choice, or a search box on particular-website (e.g. socnet, stackoverflow, reddit, github, hacker news...).
- jaytaylor 8y agoThe landing page emphasizes the word "focussed", with double S's, which didn't look right to me. Apparently both "focused" and "focussed" are acceptable, with the single S version "focused" being highly preferred [0]. [0] http://www.future-perfect.co.uk/grammar-tip/is-it-focussed-or-focused/ http://www.future-perfect.co.uk/grammar-tip/is-it-focussed-o...
- Improvotter 8y agoTIL, was about to say the same thing.
- 0xkeff 8y agohow do you solve the problem of banning / not a robot captchas after to many requests to a site while indexing?
- AJRF 8y agoExtension source code -> https://github.com/worldbrain/Memex https://github.com/worldbrain/Memex
- eloff 8y agoNice! I hacked something together to do this 12-odd years ago. I had something to save the pages I visit to folder, and then a desktop search tool dtSearch that I bought to index them. It was invaluable when I really needed it, but too awkward to be really useful. Often it ended up being easier to find the page again with Google if I remembered something unique about it. This extension looks like it could finally make it convenient enough to be more commonly useful, and privacy focused enough that I'm willing to try it. Great work! Are you planning to charge something for this in the future, or for extra features? I would definitely be willing to pay for it.
- petra 8y agoWhat about Evernote, doesn't it already do this? And how's the search quality versus this extension?
- RealWorldBrain 8y agoHey eloff, Oli here, from the team developing Memex. Memex is open-source, so the browser extension will always stay free to use. What we will charge for are some of the services that require us to host stuff. Like backups, multi-device syncing, API calls etc. We will run it as a completely modular pricing model, where you can upgrade on only those features you need. We don't like those usual 3 tier model, where you have to upgrade to the 'monster mega plan' in order to just get one feature :) But you can also completely self-host that, as we will make the server software open-source as well. Hope Memex can be useful to you. We are running a crowdfund to support its development, where we offer some good discount on the future features in return: worldbrain.io/pricing Let me know if I can be of more help.
- kovek 8y agoSeems like the Falcon extension has not gotten any updates in the last year or two: https://github.com/lengstrom/falcon https://github.com/lengstrom/falcon
- stewbrew 8y ago"Search every word of every website you visited" That will most likely return a lot of hits for sites I didn't like and would like to forget about.
- CGamesPlay 8y agoBut a normal web search already has this trait, which is presumably how you ended up on the garbage site in the first place. If you eventually found your result, a more narrow search can help you re-find it a few months down the road.
- RealWorldBrain 8y agoOli here, from the team developing Memex. Yeah indeed, just full-text search can let you end up with a lot of garbage. This is why it is so important, that you can search for various other "vague memories" to narrow down your search. What you often remember about an article is stuff like: Did I bookmark it, when did I visit it, did I like/share/cite it on social media You can already filter by time, tags, domains, bookmarks, and soon also if you liked/shared or even seen it in your newsfeed, or on a friends wall, on Twitter and Facebook. We gradually expand it so you can search with as much of your associative memories as possible.
- samat 8y agoI was imagining this for a long time. Wondering, how much more space/energy Chrome uses with this extensions enabled. Any research on that?
- samat 8y agohttps://worldbrain.io/#faq https://worldbrain.io/#faq Info on RAM/CPU/Storage, but no info on energy pressure.
- iovrthoughtthis 8y agoIve been building my own for a while. Will definitely give this a go!
- pranshuchittora 8y agofor researching on a topic and bundling them.
- fsiefken 8y agoWouldn't it be a better alternative to use an autosave plugin for the bookmarks and use a local indexer like DocFetcher or OpenSemanticSearch through the browser? Then you can also search other resources and files.
- jarsta 8y agoIt would be great to be able to change the search engine keyword from 'w' to something else. My brain has hardwired that to the Wikipedia search. Otherwise, I think this is super!
- xtiansimon 8y agoLooks like I can mark my Softwarerecs question as answered. hahah. https://softwarerecs.stackexchange.com/q/46270/16751 https://softwarerecs.stackexchange.com/q/46270/16751 While I've been waiting for this, I've been using 'Export History (2.2)' browser extension in Chrome. This saves your history to CSV or JSON.
- xtiansimon 8y agoOH! Not so fast. After a test drive, I think the Results page needs to have a button to use the same search on the web if no (useful) results are found.