This already exists in the form of ‘if-modified-since’ http cache property. A requester can ask the server if the data they currently have is stale within the c

This already exists in the form of ‘if-modified-since’ http cache property. A requester can ask the server if the data they currently have is stale within the current spec. In my experience, the problematic crawlers choose not to implement this feature.