5 ms·
> if it really was [which it isn't]
by seritools 4y ago
> if it really was [which it isn't]
- threeseed 4y agoBut many people including the OP think it is. It’s like me running a web crawler on my phone and saying I can replace Google.
- Dylan16807 4y agoIf you were able to index the same amount of content you'd have a damn good alternative. And that's the level of this experiment.
- threeseed 4y agoIt is trivial to index the same amount of content as Google as the web is mostly static. The hard part is in being able to translate a search query into a list of pages. And that requires a level of sophistication that far exceeds a laptop.
- Dylan16807 4y agoIt is not trivial at all. That is a huge amount of scraping. Any machine that can do that will be a similar spec to what you need for serving queries. Not as fast as google does it, but a good amount of them.
- threeseed 4y agoIt is trivial compared to the task of translating a user query into search results. It's akin to saying the magic behind OpenGPT is the dataset.
- Dylan16807 4y agoKeyword search is not that hard. Pagerank is not that hard. The comparable goal to the article is to be a search engine, not to fight google for best results.
- threeseed 4y ago> Keyword search is not that hard. Pagerank is not that hard. George Hotz, is that you ?
- Dylan16807 4y agoKeyword search and pagerank had working solutions decades ago. Hotz was trying to make a car controller that had never been done before, by himself, and then he wanted to """improve""" search with no explanation of what that meant that I saw. I think if he was tasked with taking twitter from no search to "has a search" he probably could have managed it. A team of five people definitely could have managed it.
- ilyt 4y agoI think you severely underestimate the effort search engines have to put to filter out all the SEO garbage and come up with some sensible results
- Dylan16807 4y agoI don't underestimate the effort, I claim that it's not strictly necessary to have more than a couple people working on it for a minimum viable product, and less for a beta.
- seritools 4y agoWhat? The OP covers much more than "just shuffling tweet sized buffers to network offload cards"
- to11mtm 4y agoMaybe it should be, though? I sometimes wonder how much value ML provides vs a proper sort function for anything but advertising.
- threeseed 4y agoIt's an easy thought experiment. Do you think that the most successful web companies in the world with arguably the best people i.e. Amazon, Facebook, Instagram, TikTok, LinkedIn, Pinterest, Youtube, Netflix, Snapchat etc. have no idea what they are doing. That the highly complex, expensive and latency impacting recommendation systems could be replaced by trivial sorting. Or maybe they do work, do translate to increased usage and do significantly impact revenue.
- fzingle 4y agoMaybe they are considering that we shouldn't build and optimize our society solely for the purposes of maximizing revenue. Would it be better to live in a world where Twitter (for example) existed because it is a useful thing and not because it might make lots of money?
- brazzledazzle 4y agoA lot of people don't realize that Dunning-Kruger can catch any of us unaware. It's easy to look at a problem, think about the surface level challenges you'd have building it and come to the conclusion that you could do it better or simpler.