3 ms·
It is trivial to index the same amount of content as Google as the web is mostly static. The hard part is in being able to translate a search query into a list
by threeseed 4y ago
It is trivial to index the same amount of content as Google as the web is mostly static.
The hard part is in being able to translate a search query into a list of pages.
And that requires a level of sophistication that far exceeds a laptop.
- Dylan16807 4y agoIt is not trivial at all. That is a huge amount of scraping. Any machine that can do that will be a similar spec to what you need for serving queries. Not as fast as google does it, but a good amount of them.
- threeseed 4y agoIt is trivial compared to the task of translating a user query into search results. It's akin to saying the magic behind OpenGPT is the dataset.
- Dylan16807 4y agoKeyword search is not that hard. Pagerank is not that hard. The comparable goal to the article is to be a search engine, not to fight google for best results.
- threeseed 4y ago> Keyword search is not that hard. Pagerank is not that hard. George Hotz, is that you ?
- Dylan16807 4y agoKeyword search and pagerank had working solutions decades ago. Hotz was trying to make a car controller that had never been done before, by himself, and then he wanted to """improve""" search with no explanation of what that meant that I saw. I think if he was tasked with taking twitter from no search to "has a search" he probably could have managed it. A team of five people definitely could have managed it.
- ilyt 4y agoI think you severely underestimate the effort search engines have to put to filter out all the SEO garbage and come up with some sensible results
- Dylan16807 4y agoI don't underestimate the effort, I claim that it's not strictly necessary to have more than a couple people working on it for a minimum viable product, and less for a beta.