3 ms·

One thing I learned from the unreliability of Google search is that we should not rely on an external service to organize our information. No matter the service

One thing I learned from the unreliability of Google search is that we should not rely on an external service to organize our information. No matter the service, it will change in some way, and you will lose access to some information.

ReactiveJelly 5y ago
Agreed. "Indexing the world's information" sounds like something that can't go wrong. It's only in the last year of trying to index and organize information within my own company that whoever controls the index holds a _lot_ of power to index it poorly.
SpelingBeeChamp 5y ago
What do you suggest as an alternative? Index the internet yourself? Web search as a public utility?
- betwixthewires 5y ago
  Well there are semantic web concepts, and there are concepts like a searchable information graph. I read a really good article on this site a few days ago laying out a pretty good architecture for searching a distributed knowledge graph. The core problem is a disconnect between the knowledge available on the internet and search tools. Indexing is a stop gap to enable the web to be usable without solving this problem, but it is becoming untenable and was always known to be non viable, just treated as a problem for another day. The sheer amount of information online and its disorganization, as well as the misaligned incentives of search tools are beginning to make this once far away manageable problem a day to day reality for all of us, the solution requires an architectural change to the way information is stored and accessed and updated online.
  - SpelingBeeChamp 5y ago
    Do you have any idea of what that solution might look like? The existence of a problem doesn't mean that there necessarily exists a better alternative.
visarga 5y ago
Why are browsers so forgetful? Local text search is an easy problem at single-user scale. I am working on a browser extension to automatically grab text (visible phrases) from the pages I am reading in the browser, save them to my index and implement search. I want to always be able to recall anything I see. In order to match texts I will use neural embeddings computed with a pretrained Huggingface model, fast enough even on CPU. An alternative way to implement this would be to take screenshots every few seconds (only after recent user interaction). Then I can OCR the images and search anything that passed on the screen, in any app. It's as if I become the Google Bot responsible with exploration and filtering of potentially useful data. In the future I might be able to use my index for neural question answering, add a voice interface to my data. Of course the local index could work for images too. You can grab images and embed them with CLIP to get Google Photos like semantic search. I think web browsers are going to morph into personal internet agents, responsible with logging, searching, interfacing with external APIs and services, filtering ads, news and mail, making sure we don't accidentally leak personal information, etc. In other words AI that works for the user, locally.
- Jonovono 5y ago
  Hey, can you email me (url in bio), would love to chat!
- hutzlibu 5y ago
  "I think web browsers are going to morph into personal internet agents, responsible with logging, searching, interfacing with external APIs and services, filtering ads, news and mail, making sure we don't accidentally leak personal information" And you think that will happen, while the browsers are still mostly add funded? But I am with you, that this is a change, I want to see, too.
  - skinkestek 5y ago
    There is an extension for Firefox (Memex Go?) that attempts something very similar and succeeds to a large degree. I think I sponsored it for a year or something by paying for it even though I didn't use it much but I must have used it to some degree because on one or more of my machines it injects interesting results from my earlier browsing when I search in DDG (or was it Google?) at least. I'm really impressed how far they've taken this idea even as Mozilla has simultaneously been busy dismantling the extension API that made Firefox great.

ReactiveJelly 5y ago

Agreed. "Indexing the world's information" sounds like something that can't go wrong. It's only in the last year of trying to index and organize information within my own company that whoever controls the index holds a _lot_ of power to index it poorly.

SpelingBeeChamp 5y ago

What do you suggest as an alternative? Index the internet yourself? Web search as a public utility?

betwixthewires 5y ago
Well there are semantic web concepts, and there are concepts like a searchable information graph. I read a really good article on this site a few days ago laying out a pretty good architecture for searching a distributed knowledge graph. The core problem is a disconnect between the knowledge available on the internet and search tools. Indexing is a stop gap to enable the web to be usable without solving this problem, but it is becoming untenable and was always known to be non viable, just treated as a problem for another day. The sheer amount of information online and its disorganization, as well as the misaligned incentives of search tools are beginning to make this once far away manageable problem a day to day reality for all of us, the solution requires an architectural change to the way information is stored and accessed and updated online.
- SpelingBeeChamp 5y ago
  Do you have any idea of what that solution might look like? The existence of a problem doesn't mean that there necessarily exists a better alternative.

visarga 5y ago

Why are browsers so forgetful? Local text search is an easy problem at single-user scale. I am working on a browser extension to automatically grab text (visible phrases) from the pages I am reading in the browser, save them to my index and implement search. I want to always be able to recall anything I see. In order to match texts I will use neural embeddings computed with a pretrained Huggingface model, fast enough even on CPU. An alternative way to implement this would be to take screenshots every few seconds (only after recent user interaction). Then I can OCR the images and search anything that passed on the screen, in any app. It's as if I become the Google Bot responsible with exploration and filtering of potentially useful data. In the future I might be able to use my index for neural question answering, add a voice interface to my data. Of course the local index could work for images too. You can grab images and embed them with CLIP to get Google Photos like semantic search. I think web browsers are going to morph into personal internet agents, responsible with logging, searching, interfacing with external APIs and services, filtering ads, news and mail, making sure we don't accidentally leak personal information, etc. In other words AI that works for the user, locally.

Jonovono 5y ago
Hey, can you email me (url in bio), would love to chat!
hutzlibu 5y ago
"I think web browsers are going to morph into personal internet agents, responsible with logging, searching, interfacing with external APIs and services, filtering ads, news and mail, making sure we don't accidentally leak personal information" And you think that will happen, while the browsers are still mostly add funded? But I am with you, that this is a change, I want to see, too.
- skinkestek 5y ago
  There is an extension for Firefox (Memex Go?) that attempts something very similar and succeeds to a large degree. I think I sponsored it for a year or something by paying for it even though I didn't use it much but I must have used it to some degree because on one or more of my machines it injects interesting results from my earlier browsing when I search in DDG (or was it Google?) at least. I'm really impressed how far they've taken this idea even as Mozilla has simultaneously been busy dismantling the extension API that made Firefox great.