If you only use a handful of sites, you can also solve this problem by building yourself a site that handles specific types of queries. I'm doing something lik

If you only use a handful of sites, you can also solve this problem by building yourself a site that handles specific types of queries. I'm doing something like this, by building a site to search lectures (https://www.findlectures.com https://www.findlectures.com). The "fallback" for me is to just use youtube, etc instead. Rather than trying to index everything on Reddit or Youtube, if you just index "good" parts it's a lot easier, since there is a lot of low quality material either way. I think you're more limited by bandwidth, for what you can get into your own index. A search index is basically a mapping of hashed search tokens -> urls, so it can be pretty efficient to store locally (e.g. for a video search engine, you just need unique words in the transcript/title, not the entire video)