Seems like the most practical path is some kind of distributed peer to peer contraption where you can allocate some storage and optionally participate in crawli

Seems like the most practical path is some kind of distributed peer to peer contraption where you can allocate some storage and optionally participate in crawling. I imagine if set some constraints you could get index size down quite a bit but I still suspect it'd be hard keeping up with content churn. Edit: Looks like enwiki bz2 is coming in around 46Gi which isn't too bad considering the amount of content it contains.