Maybe it would be nice if some sort of simple central index of "URLs + their last updated timestamp/version/eTag/whatever" would exist, updated by the site owne

Maybe it would be nice if some sort of simple central index of "URLs + their last updated timestamp/version/eTag/whatever" would exist, updated by the site owners themselves? ("push"-notification) Meaning that whenever a page of a website would be created or updated, that website itself would actively update that central index, basically saying "I just created/deleted page X" or "I just updated the contents of page X". The consequence would be that... 1) ...crawlers would not have anymore to actively (re)scan the whole Internet to find out if anything has changed, but they would only have to query that central index against their own list URLs & timestamps to find out what needs to be (re)scanned. 2) ...websites would not have to just wait&hope that some bot would decide to come by to have a look at their sites, nor they would have to answer over and over again requests that are just meant to check if some content has changed.