2 ms·
If you were able to index the same amount of content you'd have a damn good alternative. And that's the level of this experiment.
by Dylan16807 4y ago
If you were able to index the same amount of content you'd have a damn good alternative. And that's the level of this experiment.
- threeseed 4y agoIt is trivial to index the same amount of content as Google as the web is mostly static. The hard part is in being able to translate a search query into a list of pages. And that requires a level of sophistication that far exceeds a laptop.
- Dylan16807 4y agoIt is not trivial at all. That is a huge amount of scraping. Any machine that can do that will be a similar spec to what you need for serving queries. Not as fast as google does it, but a good amount of them.
- threeseed 4y agoIt is trivial compared to the task of translating a user query into search results. It's akin to saying the magic behind OpenGPT is the dataset.
- Dylan16807 4y agoKeyword search is not that hard. Pagerank is not that hard. The comparable goal to the article is to be a search engine, not to fight google for best results.
- threeseed 4y ago> Keyword search is not that hard. Pagerank is not that hard. George Hotz, is that you ?
- Dylan16807 4y agoKeyword search and pagerank had working solutions decades ago. Hotz was trying to make a car controller that had never been done before, by himself, and then he wanted to """improve""" search with no explanation of what that meant that I saw. I think if he was tasked with taking twitter from no search to "has a search" he probably could have managed it. A team of five people definitely could have managed it.
- ilyt 4y agoI think you severely underestimate the effort search engines have to put to filter out all the SEO garbage and come up with some sensible results
- Dylan16807 4y agoI don't underestimate the effort, I claim that it's not strictly necessary to have more than a couple people working on it for a minimum viable product, and less for a beta.