I built a solution at https://mitta.us/ https://mitta.us/ that lets you submit the sites you want crawled, and puts them in a self-managed index (which isn't sh

I built a solution at https://mitta.us/ https://mitta.us/ that lets you submit the sites you want crawled, and puts them in a self-managed index (which isn't shared globally). I don't do link extraction, but instead let GPT-3 generate URLs based off keywords. !url <keyterms> |synthesize I also wrote a screenshot extension for Chrome that lets you save a page when you find it interesting. The site is definitely not "done" but it's usable if you want to try it. Some info in help and in commands is inaccurate/broken, so it is what it is for now. It does the !google <search term> and !ddg <search term> thing to find pages to save to the index. There are a bunch of other commands I added, and there's an ability for others to write commands and submit them to a Github repo: https://github.com/kordless/mitta-community https://github.com/kordless/mitta-community !xkcd was fun to write. It shows comics. The rest of the commands can be viewed from !help or just !<tab> I've been working on pivoting the site to do prompt management for GPT-3 developers and have been kicking around Open Sourcing the other version for use as a personal search engine for bookmarked pages.