3 ms·
I don't get it... if News Corp doesn't want to be indexed can't they just set their robots.txt accordingly? It doesn't seem like they'll be able to successfully
by 3131s 6y ago
I don't get it... if News Corp doesn't want to be indexed can't they just set their robots.txt accordingly? It doesn't seem like they'll be able to successfully argue this.
- brainwad 6y agoWhat this law gives them is a way to force all their competitors (e.g. Fairfax) to join their boycott, by being able to negotatiate a single, mandatory price for the industry that their competitors won't be allowed to undercut.
- Strom 6y agoI'll just note that robots.txt does not affect indexing. It is used for traffic management and will not hide pages from Google. [1] -- [1] https://support.google.com/webmasters/answer/6062608?hl=en https://support.google.com/webmasters/answer/6062608?hl=en