3 ms·
It is already. And in addition to that, since 'robots.txt' is being considered to broad and too 'out of time', Google offers meta tags, that their bot evaluates
by zmix 7y ago
It is already. And in addition to that, since 'robots.txt' is being considered to broad and too 'out of time', Google offers meta tags, that their bot evaluates. They did this long time before the EU direction.