While I agree, opt-in would certainly be better.. I really see no way to transition from the current state, to 100% opt in. And, I think opt-out was the right

While I agree, opt-in would certainly be better.. I really see no way to transition from the current state, to 100% opt in. And, I think opt-out was the right choice at the beginning - nobody would have updated their sites to suit the startup search engine. That said, Google & co could set a date, whereby new content / pages will not be indexed unless its marked as indexable. This would allow historic content to remain indexed, and new content be opt-in at the expense of any new search engines being unable to index historic content. How can they tell if the content has not opted in, or is just no longer actively maintained without having a pre-existing index?